-- Logs begin at Thu 2022-07-21 11:32:10 UTC, end at Thu 2022-07-21 13:01:39 UTC. -- Jul 21 11:59:58.727975 dv-cinder-850321-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 11:59:58.738926 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 11:59:58 2022] *** Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: nodename: dv-cinder-850321-1-iscsi Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: machine: x86_64 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: clock source: unix Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: pcre jit disabled Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: detected number of CPU cores: 4 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: current working directory: / Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: detected binary path: /usr/bin/uwsgi-core Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: your processes number limit is 47680 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: your memory page size is 4096 bytes Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: detected max file descriptor number: 2048 Jul 21 11:59:58.741035 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: lock engine: pthread robust mutexes Jul 21 11:59:58.741928 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: thunder lock: enabled Jul 21 11:59:58.741928 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 11:59:58.741928 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 11:59:58.784264 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: Python main interpreter initialized at 0x56360eae19c0 Jul 21 11:59:58.784264 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: python threads support enabled Jul 21 11:59:58.784264 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: your server socket listen backlog is limited to 100 connections Jul 21 11:59:58.784264 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: your mercy for graceful operations on workers is 90 seconds Jul 21 11:59:58.784634 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 11:59:58.784697 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: *** Operational MODE: preforking *** Jul 21 11:59:58.784971 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: *** uWSGI is running in multiple interpreter mode *** Jul 21 11:59:58.784971 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: spawned uWSGI master process (pid: 80373) Jul 21 11:59:58.785217 dv-cinder-850321-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 11:59:58.786210 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: spawned uWSGI worker 1 (pid: 80383, cores: 1) Jul 21 11:59:58.786210 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: spawned uWSGI worker 2 (pid: 80384, cores: 1) Jul 21 11:59:58.787150 dv-cinder-850321-1-iscsi devstack@keystone.service[80373]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 12:00:04.181272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 12:00:04.181272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: from cryptography.utils import int_from_bytes Jul 21 12:00:04.182094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 12:00:04.182094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: from cryptography.utils import int_from_bytes Jul 21 12:00:04.224741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 12:00:04.224741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: from cryptography.utils import int_from_bytes Jul 21 12:00:04.225381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 12:00:04.225381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: from cryptography.utils import int_from_bytes Jul 21 12:00:04.230568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 12:00:04.230748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 12:00:04.230980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] command line args: [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 12:00:04.231255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 12:00:04.231415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ================================================================================ {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 12:00:04.231728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] admin_token = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.231977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] config_dir = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.232194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.232373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] config_source = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.232584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] debug = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.232803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.233152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_config_append = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_dir = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.234858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_file = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.235114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.235371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.235663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.235892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.236072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.236249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.236492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.236737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.236971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.237214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.237473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.237708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.238002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.238244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.238528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.239042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.239232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] publish_errors = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.239494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.239804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.239992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.240241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.240477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.240712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] standard_threads = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.240944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.241193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.241370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.241547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] use_journal = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.241723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] use_json = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.241899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] use_stderr = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.242075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] use_syslog = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.242307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.242564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.242801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.243060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.243301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.243573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.243838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.244102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.244392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.244654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.244952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.245217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.245501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.245926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.246204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.246597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.246886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.247266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.247566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.247843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.248111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.248492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.248787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.249069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.249351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.249628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.249886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.250148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.250405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.250673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.250952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.251239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.251515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.251782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.252059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.252388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.252656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.252999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.253237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.253480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.253715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.253946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.254175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.254405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.254634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.254870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.255118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.255349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.255608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.255841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.256077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.256322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.256563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.256793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.257031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.257268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.257506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.257743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.257981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.258212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.258462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.258698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.258934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.259163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.259422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.259684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.259915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.260154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.260389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.260659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.260895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.261148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.261451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.261685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.261919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.262153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.262398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.262635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.262869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.263110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.263357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.263612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.263845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.264081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.264315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.264575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.264814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.265053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.265293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.265539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.265777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.266011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.266245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.266478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.266707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.266935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.267167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.267396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.267659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.267894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.268141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.268378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.268635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.268894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.269133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.269364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.external = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.269593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.269888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.270117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.270347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.password = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.270576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] auth.token = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.270816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.271054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.271246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 12:00:04.271328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.271450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 12:00:04.271557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.271670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] command line args: [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 12:00:04.271791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.271941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 12:00:04.272021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.272119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ================================================================================ {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 12:00:04.272222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.272370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] admin_token = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.272449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.272617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] config_dir = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.272694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.272804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.272955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.273030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] config_source = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.273151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] debug = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.273226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.273363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.273480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.273600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.273681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.273779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.273863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.274029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.274115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.274277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.274354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.274507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.274583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.274687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_config_append = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.274772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.274865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.275043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_dir = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.275118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.275221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_file = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.275298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.275458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.275607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.275736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.275839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.276032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.276134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.276267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.276381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.276521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.276627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.276751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.276837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.276962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.277036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.277223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.277304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.277469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.277555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.277705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.277789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.277957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.278028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.278192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.278270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.278453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.278544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.278693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.278775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.278943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.279061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.279180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.279427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.279557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.279712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.279802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] publish_errors = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.279953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.280032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.280191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.280307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.280413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.280578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.280666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.280815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.280917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.281064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.281173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] standard_threads = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.281319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.281406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.281571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.281673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.281805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.281889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.282121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] use_journal = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] use_json = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.282421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] use_stderr = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.282628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] use_syslog = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.282822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 12:00:04.282971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.283981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.284908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.285919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.286830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.287951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.288980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.289937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.290894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.291964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.292963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.293941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.294961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.295914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.296862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.297975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.298980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.299945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.300914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.301861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] role.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.302971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] role.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.303943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.304921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.305929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.306975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.307973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.external = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.308960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.password = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.309982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] auth.token = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.310930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.311819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.312865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.313973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.314943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315511 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.315943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.connection = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.316991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.317929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.318876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.319845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80383) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 12:00:04.320612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.320870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.321107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.321342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.321576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.321825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.322074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.322325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.322585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.322835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.323110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.323352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.323621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.323914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.324157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.324390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.324630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.324876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.325120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.325374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.325607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.325855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.326112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.326352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.326597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.326846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.327085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.327322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.327596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.327824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.328070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.328320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.328559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.328799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.329053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.329305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.329538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.329770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.330017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.330274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.330515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.330754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.330990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.331222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.331463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.331753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.331995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.332245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.332496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.332746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.332980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.333212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.333451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.333686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.333923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.334154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.334399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.334648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.334895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.335142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.335386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.335642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.335892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.336130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.336368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.336608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.336842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.337088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.337685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.337931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.338202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.338417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.338662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.338904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.339142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.339412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.339682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.339932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.340162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.340396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.340639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.340889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.341141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.341378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.341628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.341880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.342114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.342362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.342600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.342851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.343091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] role.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.343342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] role.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.343609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.343865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.344113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.344363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.344614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.344862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.345118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.345369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.345618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.345846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.346096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.346344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.346589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.346839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.347268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.347518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.347780 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.348030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.348282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.348532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.348769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.349034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.349288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.349530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.349763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.349999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.350237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.350492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.350750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.350998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.351236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.351474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.351653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.351772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.351902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.352030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.352154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.352277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.352406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.352641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.352899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.353149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.353390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.353635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.353881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.354131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.354371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.354611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.354848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.355151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.355425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.355671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.355916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.356227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.356490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.connection = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.356741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.356989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.357232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.357476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.357712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.357950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.358191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.358429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.358677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.358920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.359167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.359404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.359683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.359921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.360168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.360380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.360518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.360675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.360850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 12:00:04.361020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 12:00:04.387170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.387373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.387712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.388347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.388580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.396303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.414340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.414509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.537779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.547152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.593266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.593502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.593697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.593925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.664991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.665667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.691946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.692855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.704266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.704432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.717733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.717918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.718996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.719076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.719204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.719294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80383) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 12:00:04.736245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.736490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.736588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 12:00:04.736717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 12:00:04.736821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.736948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 12:00:04.737147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 12:00:04.737264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.737894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.738014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.738105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.749854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.750077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.757928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.758294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.761282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.761566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.761691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.761847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.761990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.762983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.763961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.764300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.764423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.764423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.792372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.792564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.794134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.794298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.794427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.794599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.799766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.800009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.800135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.800296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.800443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.800526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.807612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.807902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.808026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.808290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.812651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.812825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.812953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.813928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.814905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.833243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.833517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.834062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.834340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.839145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.839357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.842882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.843115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.843240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.843412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.848229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.848418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.851887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.852013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.855696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.855842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.855973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.856805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.857022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.857150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.857284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.876065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.876358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.876512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.876645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.876793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.876916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.877040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.877165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.884849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.885067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.886569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.886857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.886995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.887108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.889932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.890190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.890319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.890458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.890545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.890736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.890932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.891028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.891102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.891320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.904481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.904745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.904871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.905877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.906002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.906096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.906173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.906415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.925614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.925836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.928833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.929138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.932170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.932481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.933817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.934115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.935450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.935751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.935874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.936045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.936168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.936335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.943696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.944991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.945372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.947734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.948598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.948958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.949245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.949512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.951018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.951317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.951632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.951915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.958598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.958954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.959389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.961113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.961409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.961695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.967759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.968035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.968305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.968582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.968865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.969132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.969411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.969694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.969938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80383) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 12:00:04.970025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.970862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.971886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.972162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.972432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.972706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.972968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.973244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:04.973519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80383) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 12:00:05.010100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56360eae19c0 pid: 80384 (default app) Jul 21 12:00:05.011512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56360eae19c0 pid: 80383 (default app) Jul 21 12:00:05.013596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7a60-5da4-4855-b2f7-29f2c40f7acc None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:05.013892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7a60-5da4-4855-b2f7-29f2c40f7acc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:05.014173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c7a60-5da4-4855-b2f7-29f2c40f7acc None None] PATH_INFO: `/v3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:05.018524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Thu Jul 21 12:00:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:12.789893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:12.790574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:12.790574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:12.794865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:12.800517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:12.800980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:12.801072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:12.870217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: INFO dbcounter [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] Registered counter for database keystone Jul 21 12:00:12.891922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG oslo_db.sqlalchemy.engines [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80384) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 12:00:12.898957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writer thread running {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 12:00:13.079411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG passlib.handlers.bcrypt [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80384) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 12:00:13.079764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG passlib.handlers.bcrypt [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80384) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 12:00:13.135366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:13.137095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:13.270447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 12:00:13.449461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:13.519944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: INFO dbcounter [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] Registered counter for database keystone Jul 21 12:00:13.541374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG oslo_db.sqlalchemy.engines [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80383) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 12:00:13.549378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writer thread running {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 12:00:13.667471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7524b3-79f2-4f48-bb29-86d16859de7f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:13.673268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:13.674295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:13.674748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:13.675199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:13.686821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:13.738191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING keystone.common.rbac_enforcer.enforcer [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 12:00:13.738449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae26f-b477-4ed7-bdc9-bd88bda2c7c2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:13.749122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Jul 21 12:00:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:00:13.755517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:13.770362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504dd75-4140-46d8-9733-0cf6642b1742 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:13.774747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:13.775920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:13.776210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:13.776484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] PATH_INFO: `/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:13.780766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] RBAC: Authorizing `identity:get_project(project_id=8a4579ac3a4640e986aabfbb2b1ede46)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:13.830495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING keystone.common.rbac_enforcer.enforcer [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 12:00:13.831107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c1756-20f1-4b92-8af1-34bdcb9c609c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:13.835554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Thu Jul 21 12:00:13 2022] GET /identity/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:15.869918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f573-e3d4-4a1a-a6d4-0080fc611f75 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:15.870510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f573-e3d4-4a1a-a6d4-0080fc611f75 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:15.870510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4f573-e3d4-4a1a-a6d4-0080fc611f75 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:15.874663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:00:15.880367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:15.880793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:15.880883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:15.903445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:15.905060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:15.915016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 12:00:15.937694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4f573-e3d4-4a1a-a6d4-0080fc611f75 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:15.952021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4f573-e3d4-4a1a-a6d4-0080fc611f75 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:15.957217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:15.957868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:15.958164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:15.958434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:15.967438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:15.969255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff8291-a54e-4756-abe7-e4f8e95e0115 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:15.978764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Thu Jul 21 12:00:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:00:15.985231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:15.995459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca314-d348-48f1-98ce-41a25df657b9 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:16.001416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:16.002319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:16.002602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:16.002873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:16.004510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:16.006397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cca34-4ada-40d3-967f-046c96d73749 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:16.008075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Thu Jul 21 12:00:15 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:18.158115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d51bf-e288-450b-91a8-63605823e956 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.158483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d51bf-e288-450b-91a8-63605823e956 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.158670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d51bf-e288-450b-91a8-63605823e956 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.160085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:18.161084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138365c-723a-466e-b225-d61bb5e7f551 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.161448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138365c-723a-466e-b225-d61bb5e7f551 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.161641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138365c-723a-466e-b225-d61bb5e7f551 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.163024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:18.165639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.165944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.166225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.168525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.168817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.169195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.186091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:18.187324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:18.208766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG passlib.handlers.bcrypt [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80383) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 12:00:18.209161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG passlib.handlers.bcrypt [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80383) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 12:00:18.254742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:18.256358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:18.273705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 7/12] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:18.297216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.324826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:18.372275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 6/13] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:18.396762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.416003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c9f0e-1fbf-466c-ba00-e228757ac361 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.453957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.454783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.455065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.455339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.469045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:18.480897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:18.481873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:18.483057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 8/14] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:00:18.489162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.512168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f11a2-cb23-4e86-bb7d-36d2b017b9b4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.515334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:18.561507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.562450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.562732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.563009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.580154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:18.600078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcc822-3d84-4024-9272-f1c9e715feef None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.637064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:18.638929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:18.646021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.646706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.646987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.647258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.648077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:18.648399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:18.649825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:18.662689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 9/15] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:18.668572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.769894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 7/16] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 12:00:18.774064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b84bf-f1c3-4c6a-80b0-8a8b6c4ffa53 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.776617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.780107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:18.811616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.812510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.812794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.813067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.814342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:18.815895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:18.818082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 10/17] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:18.823845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.839462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:18.894633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e1893-43f7-4eff-bf27-a3331d91ddcb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.933479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750f623-c466-4418-af19-e0f9f8e35da2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:18.941451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.942846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.943349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.945025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.971904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:18.972734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:18.973046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:18.973297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:18.974525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:18.976167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:18.980781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 11/18] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:18.984113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:18.984908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:18.985643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:00:19.007300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:19.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:19.010071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:00:19.010071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:00:19.010071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:00:19.010071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:19.010071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 8/19] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:00:19.014976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:19.088715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:19.089645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:19.111903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f077-4133-4bca-8021-a9c6d0e088e2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:19.136744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f95768-efbf-4224-94db-729df17bfb19 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:19.148875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:19.148875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:19.148875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:19.148875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:19.152455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.domain.id=default, user.id=7aeac511a0044d6781fee6120328a71c) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:19.152455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:19.152805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d280bfd-e102-4aab-80cf-cad86a94d0a8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:19.161227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 12/20] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Jul 21 12:00:18 2022] GET /identity/v3/role_assignments?user.id=7aeac511a0044d6781fee6120328a71c&scope.domain.id=default&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 233 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:19.178265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:19.178991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:19.179275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:19.179592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:19.180373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:19.180681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:19.182172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2770b9-0f1e-4d11-a4d0-b5d7d7f228b4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:19.203766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 9/21] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:21.650962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7fa6-117d-4744-a37d-64e2537defd2 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:21.651810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7fa6-117d-4744-a37d-64e2537defd2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:21.652072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd7fa6-117d-4744-a37d-64e2537defd2 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:21.654770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 13/22] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:21.673392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:21.674141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:21.674141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:21.701024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:21.702240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:21.804449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 10/23] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:21.847883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd7fa6-117d-4744-a37d-64e2537defd2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:22.124061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd7fa6-117d-4744-a37d-64e2537defd2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:22.199131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:22.199976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:22.200263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:22.200542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:22.217713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:22.225481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:22.227062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:22.228634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 14/24] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:22.232993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:22.303377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:22.381299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8e59a-c69d-4ece-8df8-108db92274b7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:22.423402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:22.424154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:22.424445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:22.424727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:22.425516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:22.425827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:22.427302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:22.445805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 11/25] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:22.452122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:22.456849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:22.582615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0acf0-e5b0-40c3-8c63-3245c0ffb2f7 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:22.627178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:22.628116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:22.628389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:22.628664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:22.632403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:22.633843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da99198-ebf2-4cb7-b74e-ca9dc6e860ca None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:22.636010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 15/26] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:22 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:22.641774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:22.701518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d02c6-675a-47f6-a16f-f10f8db00a3b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:22.702319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d02c6-675a-47f6-a16f-f10f8db00a3b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:22.702319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d02c6-675a-47f6-a16f-f10f8db00a3b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:22.703816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 16/27] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:22.719079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:22.719458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:22.719732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:22.747978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:22.749263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:22.896624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c71cf-c41c-4168-afc9-6ab600674a91 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:22.922991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:22.934595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 17/28] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:22.969625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:22.971928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:23.012744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:23.014132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.014637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.015132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.017995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:23.025343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:23.029507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 12/29] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:23.037267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:23.258059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42284175-c439-46f7-81b6-1dbd4c0d29aa None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:23.268828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3969-4d41-4531-9fa4-594ee2829531 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:23.313093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:23.314687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.314801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:23.315173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.315681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] PATH_INFO: `/v3/domains/default/users/7aeac511a0044d6781fee6120328a71c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.316011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.316434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.316964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.317887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:23.319581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f6802-5246-4c66-b271-4f7f87b323ee None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:23.319768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7aeac511a0044d6781fee6120328a71c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:23.322687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66ae5d-6a6c-4446-b702-843b76360ac4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:23.461649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 18/30] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Jul 21 12:00:22 2022] POST /identity/v3/roles => generated 225 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:23.463526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bb8-ea72-49a0-9496-cd0a834e95b1 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.463831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bb8-ea72-49a0-9496-cd0a834e95b1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.464185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1130bb8-ea72-49a0-9496-cd0a834e95b1 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.465501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 19/31] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:23.466678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c10b-88f8-45ec-98f1-643e90428418 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.466973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c10b-88f8-45ec-98f1-643e90428418 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.467254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8c10b-88f8-45ec-98f1-643e90428418 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.468537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 20/32] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:23.469697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c189aa-e247-4bfa-82a6-7c7186e81eed None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.470036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c189aa-e247-4bfa-82a6-7c7186e81eed None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.470309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c189aa-e247-4bfa-82a6-7c7186e81eed None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.471600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 21/33] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:23.479574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b03d9-414a-4b4d-a97c-739cb8de3a21 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.479574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b03d9-414a-4b4d-a97c-739cb8de3a21 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.479574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b03d9-414a-4b4d-a97c-739cb8de3a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.499422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e67b03d9-414a-4b4d-a97c-739cb8de3a21 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:23.501235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e67b03d9-414a-4b4d-a97c-739cb8de3a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:23.534827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 13/34] 10.222.0.18 () {58 vars in 1361 bytes} [Thu Jul 21 12:00:23 2022] PUT /identity/v3/domains/default/users/7aeac511a0044d6781fee6120328a71c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:00:23.536348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.536633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.536917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.557703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:23.559678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:23.588226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:23.598183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 22/35] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:23.600538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.600971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.601356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.636517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:23.649422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:23.651640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:23.692818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 14/36] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:23.693724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:23.830239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7025-be12-4d93-b024-1d8750ccad3a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:23.891592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 23/37] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:23.892611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:23.893375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:23.895254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:23.896159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:23.896943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:23.914832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:23.918897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:00:23.952051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:23.953596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:23.955447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:00:23.955447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:00:23.955447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:00:23.955447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:23.959517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 15/38] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:23.961542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:24.004580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:24.016830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28270a1-320c-4e40-84b5-17d074b60203 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:24.044513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:24.060153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:24.060970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.061354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.061710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.062576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:24.064306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:24.084844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 24/39] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Jul 21 12:00:23 2022] POST /identity/v3/roles => generated 219 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:24.085853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:24.189010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc086f6e-cea3-448c-892c-e951394718d2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:24.280848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:24.281581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.281868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.282155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.282980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:24.284554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:24.315873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1508f9c-046d-4bdb-a7c0-5c09282665ce None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:24.321006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 16/40] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Jul 21 12:00:23 2022] POST /identity/v3/roles => generated 223 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:24.373857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:24.374542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.374833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.375139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.376370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:24.376706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:24.378688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5ddbd-3aa2-473b-bb38-8ff48ab1fc53 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:24.406982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 25/41] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:24.415331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:24.560259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74254-426b-4db8-a600-d3380a819286 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.560586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74254-426b-4db8-a600-d3380a819286 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.560865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac74254-426b-4db8-a600-d3380a819286 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.562535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 26/42] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:24.569580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.569920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.570189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.577369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:24.616362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7141ac6-7aa2-425d-b21e-bde7eae44e6c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:24.683107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:24.684358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:24.684863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:24.685367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:24.686827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:24.689470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:24.731356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:24.732686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:24.743527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 17/43] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:00:24 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:24.777067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:24.821340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 27/44] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:24.846106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:25.017682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273c111-dace-4a07-9d7b-a2cc8dc119dc None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:25.054497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:25.062511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:25.063365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:25.063688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:25.063972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:25.065552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:25.067087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c1e0b-f688-4b18-ba06-8fcc10a638c8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:25.068914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 18/45] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:25.075114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:25.184996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497b2c6-701d-478b-9b8c-c35dbda7e49c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:25.220924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:25.221642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:25.221920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:25.222246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:25.222961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:25.224573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f612d9-7fb8-4d30-908a-226586fd6f00 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:25.252746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 28/46] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:00:25 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:25.935072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b2125-2e63-4275-835a-4842f54f7b7a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:25.935366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b2125-2e63-4275-835a-4842f54f7b7a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:25.935675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b2125-2e63-4275-835a-4842f54f7b7a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:25.937077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 19/47] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:25.942527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:25.942813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:25.943092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:25.960842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:25.962413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:25.981576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:26.036155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 29/48] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:26.059246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b2125-2e63-4275-835a-4842f54f7b7a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:26.169197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b2125-2e63-4275-835a-4842f54f7b7a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:26.204894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:26.205613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:26.205879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:26.206154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:26.218922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:26.227703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:26.228648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:26.229374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 20/49] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:26.238045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:26.342414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23e459-7acc-44ce-9b4d-e7eedb344d67 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:26.376078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:26.376757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:26.377036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:26.377308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:26.378080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:26.378384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:26.379863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:26.390306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 30/50] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:26.397129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:26.497707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:26.541472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1e259-97af-4456-9e06-ccca5c738137 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:26.578645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:26.579844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:26.580247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:26.580527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:26.581852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:26.585768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:26.585768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 21/51] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:26 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:26.591754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:26.611651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:26.693865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade35f67-68db-4c90-aa68-3592be33b23e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:26.728209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:26.729009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:26.729287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:26.729555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:26.730806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:26.732308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fe189-b124-4c7d-9cd2-8ba40d1d28d2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:26.733954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 31/52] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:26.740018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:26.844316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2425ed-75d0-43cf-ad38-9f5ac1872cd4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:26.871624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:26.881672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:26.882353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:26.882639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:26.882916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:26.883814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.domain.id=default, user.id=7aeac511a0044d6781fee6120328a71c) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:26.884129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:26.885614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd991e2-67b3-4bb5-8145-32ca4bcc1144 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:26.894608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 22/53] 10.222.0.18 () {56 vars in 1322 bytes} [Thu Jul 21 12:00:26 2022] GET /identity/v3/role_assignments?user.id=7aeac511a0044d6781fee6120328a71c&scope.domain.id=default&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:27.317893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c9a72-6c4e-412a-a061-528d541078f7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:27.318468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c9a72-6c4e-412a-a061-528d541078f7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:27.318468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c9a72-6c4e-412a-a061-528d541078f7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:27.319840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 32/54] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:27.325555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477180-1326-4622-9e07-80b243995d65 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:27.325991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477180-1326-4622-9e07-80b243995d65 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:27.326136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477180-1326-4622-9e07-80b243995d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:27.458158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-00477180-1326-4622-9e07-80b243995d65 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:27.459330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-00477180-1326-4622-9e07-80b243995d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:27.529871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 23/55] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:27.553562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c9a72-6c4e-412a-a061-528d541078f7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:27.658558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:27.677761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c9a72-6c4e-412a-a061-528d541078f7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:27.714531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:27.715249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:27.715583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:27.715822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:27.717082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:27.718570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b1240-a182-4f71-a625-13ee9a5c4150 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:27.720242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 33/56] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:27.726173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477180-1326-4622-9e07-80b243995d65 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:27.832859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477180-1326-4622-9e07-80b243995d65 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:27.854402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:27.869421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:27.870097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:27.870527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:27.870810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:27.871586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:27.873120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c29a2-69bd-4551-b6be-e88c3ca33846 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:27.900733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 24/57] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Jul 21 12:00:27 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:29.966452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab88f-7a23-4d25-8997-ff000506aa32 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:29.967038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab88f-7a23-4d25-8997-ff000506aa32 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:29.967038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab88f-7a23-4d25-8997-ff000506aa32 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:29.968434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 34/58] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:29.974119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:29.974557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:29.974674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:29.992587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:29.993781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:30.064909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 25/59] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:30.089977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab88f-7a23-4d25-8997-ff000506aa32 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:30.229213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab88f-7a23-4d25-8997-ff000506aa32 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:30.268215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:30.269022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:30.269307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:30.269608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:30.270901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:30.272385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211eb61-34e0-486d-a615-ad381b900abf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:30.274014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 35/60] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:30.281030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:30.393885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa28ed-9c8f-43d0-9a4d-d1648dda2e00 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:30.395524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:30.430343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:30.431029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:30.431320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:30.431634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:30.432377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:30.433889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4756e-fea2-4fd9-b5c2-680522d048de None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:30.462229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 26/61] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:30 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:00:33.866322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cf56-1f8b-4acc-921f-201bbb341c5a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:33.867948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cf56-1f8b-4acc-921f-201bbb341c5a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:33.868770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cf56-1f8b-4acc-921f-201bbb341c5a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:33.871786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 36/62] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:33.877531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:33.878260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:33.878260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:33.906317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:33.908377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:34.008889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 27/63] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:34.043201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6c21-d6fb-4112-b475-1e15d044bc04 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.044096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6c21-d6fb-4112-b475-1e15d044bc04 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.044552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.045133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6c21-d6fb-4112-b475-1e15d044bc04 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.047929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 37/64] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:34.063387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4da15-fd32-40f3-b32e-13b63bf88966 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.064042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4da15-fd32-40f3-b32e-13b63bf88966 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.064851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4da15-fd32-40f3-b32e-13b63bf88966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.107679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-36b4da15-fd32-40f3-b32e-13b63bf88966 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:34.110208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-36b4da15-fd32-40f3-b32e-13b63bf88966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:34.156893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.196053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.229837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba850fb6-2fa6-443d-ba81-0937407c286d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.285440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 38/65] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:34.306765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:34.308271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.308744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.309206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.314152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f23ba4-e7c3-4ae3-9be0-4b2b72aebe6b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.314636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f23ba4-e7c3-4ae3-9be0-4b2b72aebe6b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.315087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f23ba4-e7c3-4ae3-9be0-4b2b72aebe6b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.316672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 39/66] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:34.322902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.323546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.324053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.329715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:34.342843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:34.344463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:34.346949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:34.347841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:34.348577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:34.349282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 28/67] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:34.350894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.431623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 40/68] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:34.431623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.532554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.558810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f765a-907e-4717-ac1d-e09628d21a47 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.560888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272a08e-62e0-4555-b527-358bf85b9113 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.603325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:34.604274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.604695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.605392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.618039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:34.623613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:34.624909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.625542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.626142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:34.627604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:34.628439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:34.629173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:34.629724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:34.629724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:34.631632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:34.637146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 29/69] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:34.638403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.648729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 41/70] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:34.650631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.651556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.759225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84c03-7364-4ced-8b24-27d7e14dd821 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.795093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:34.795962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.796252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.796537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.804821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df835e1-b3cd-4054-849d-1a6ab6c95416 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.808504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:34.816439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:34.818222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 30/71] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:34.819727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.846642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:34.847334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.847665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.847944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.848722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:34.849026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:34.850490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c736d7-42d6-4b37-8611-173f5ce1936e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 42/72] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1d5b-c6a2-4a8d-8463-09a33bd14545 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1d5b-c6a2-4a8d-8463-09a33bd14545 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1d5b-c6a2-4a8d-8463-09a33bd14545 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 43/73] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d4bae-c58b-47c0-9233-89cfc2626780 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:34.866193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d4bae-c58b-47c0-9233-89cfc2626780 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:34.867824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d4bae-c58b-47c0-9233-89cfc2626780 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:34.867824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 44/74] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:34.867824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d4bae-c58b-47c0-9233-89cfc2626780 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:34.874548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.906166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:34.955274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3fdb3-3de0-41bf-b697-7d35f7f575a7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:34.991836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d4bae-c58b-47c0-9233-89cfc2626780 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.001337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.002877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.003384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.003917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.029134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.029720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.030009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.030288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.031070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:35.031381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.031604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.032851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8eebd-ea0f-4c4e-9775-1c05b06c86f1 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.034399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.037898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 31/75] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:35.038864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.047023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 45/76] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:35.049368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008843-e306-4d81-b05f-4edc9921d24a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.049790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008843-e306-4d81-b05f-4edc9921d24a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.050209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008843-e306-4d81-b05f-4edc9921d24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.069674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fb008843-e306-4d81-b05f-4edc9921d24a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:35.071106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fb008843-e306-4d81-b05f-4edc9921d24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:35.139552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.152937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 46/77] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:35.154855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.155287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.155774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.175840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:35.176088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691cdf-9c24-4b4e-9281-fd27bc941974 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.177320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:35.207340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.221077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.222640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.223145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.223679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.225965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.228911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.232569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 32/78] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:35.233775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.262016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 47/79] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:35.262943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.366219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ea842-cdf1-4feb-81b5-9bafe1e4160b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.402671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.403655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.404048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.404554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.415072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701e38d-3eca-4763-8951-7df0ef5fd8cc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.443880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.454080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.455021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.455308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.455621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.456919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.458422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.460497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 48/80] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:35.461061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.487327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.496053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.497572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.591230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c0a0-67ca-4445-b7f1-98d6fa47844b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.601784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 33/81] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:35.602663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.629867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.630593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.630879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.631144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.632429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.633924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.635604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 49/82] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:35.636489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.715187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.724969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.736003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89453-0510-4e2f-8271-b1bf61503d20 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.759641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f996a-cddb-4892-99c2-51b4cc02f1cf None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.775508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.776176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.776506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.776789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.808467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.809287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.809572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.809845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:35.822814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:35.830729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:35.831644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:35.832388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 50/83] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:35.833084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.856817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:35.858347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:35.940461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b35d4-97a1-4209-a03c-56dfa6656713 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:35.966100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.968904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:35.976108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 34/84] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:35.976963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:35.980750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:35.981423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:35.981712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:35.981994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.069873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.071402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.094293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea2355-327f-4f56-8308-79bbcbd51a69 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.133419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.134490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.134991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.135499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.137589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.140382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.143653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 35/85] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.144976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.186591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 51/86] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:36.191582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.216086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:36.251802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:36.301750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be144d0-a58d-4389-8a68-6e505726dd81 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.305403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d942a-1772-4e1b-91be-62d779459c3c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.338591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.339365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.339682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.339942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.340728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:36.341026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.342539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.345901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.346595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.346883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.347164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.347947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.349445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.352152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 52/87] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.355812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.378201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 36/88] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:36 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:36.380042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.483856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:36.506088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fda33-14b5-488f-8093-0d3065296df0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.524198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473662d-050f-4ee6-852d-9d9783e82872 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.545378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:36.551292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.551292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.551733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.551861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.553083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.554575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.556408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 53/89] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.557235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.581991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.582685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.582965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.583330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.584132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:36.584449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.585957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.597860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 37/90] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.603581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.733393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f7d2-8715-427d-a969-ddcee2e995ef None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.779236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34498b9e-a3f5-45f3-99bd-574e96d53274 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.785565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.786702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.786986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.787264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.789870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.791870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.795137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 54/91] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.796029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.829257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.830161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.830443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.830717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.832147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.833650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.835402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:36.836469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 38/92] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.837997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.926647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdecf1b-cc14-40f8-89d2-d3cdcd2fb5ab None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:36.964204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:36.964958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:36.965232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:36.965526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:36.966460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=7aeac511a0044d6781fee6120328a71c) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:36.966783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:36.968402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:36.975088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 55/93] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/role_assignments?user.id=7aeac511a0044d6781fee6120328a71c&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:36.975816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:36.987902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:37.049124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a0e58-0745-4599-9d06-e4967520a92a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:37.094974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:37.095924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:37.096216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:37.096495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:37.097421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:37.097728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:37.099323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:37.104789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:37.115715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 39/94] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:37.132617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cfe8d-3446-4672-bdd7-2030993dbea0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:37.203679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:37.204398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:37.204688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:37.205014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:37.292822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:37.294473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:37.399948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:37.420612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 56/95] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:37.426493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:37.563010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f2f5-6ccc-4218-b75c-b03d592a1525 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:37.584988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:37.609316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:37.610212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:37.610496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:37.610769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] PATH_INFO: `/v3/projects/ba6c02055d0b42aab693c168e4f2d58b` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:37.612097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ba6c02055d0b42aab693c168e4f2d58b)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:37.613571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728816f-3291-4b9c-beb8-a6c3235ce8b0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:37.615586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 40/96] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:37 2022] GET /identity/v3/projects/ba6c02055d0b42aab693c168e4f2d58b => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:37.624664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:37.779347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d8595-3c8e-4225-b7f5-10fdc8dff101 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:37.820727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:37.821454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:37.821741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:37.822022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:37.822985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=ba6c02055d0b42aab693c168e4f2d58b, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:37.823394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:37.825015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbbaaa-c89a-4d7c-bbe2-6a6c5a5e9824 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:37.831978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 57/97] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:37 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=ba6c02055d0b42aab693c168e4f2d58b&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:39.272957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254f6e-9a44-4dc7-b808-a20ae493dad3 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.273474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254f6e-9a44-4dc7-b808-a20ae493dad3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.273474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23254f6e-9a44-4dc7-b808-a20ae493dad3 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.275122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 41/98] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:39.281145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3269-dcec-43f1-937d-9af2cc7c929e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.282106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3269-dcec-43f1-937d-9af2cc7c929e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.282581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3269-dcec-43f1-937d-9af2cc7c929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.308047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ffec3269-dcec-43f1-937d-9af2cc7c929e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:39.309763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ffec3269-dcec-43f1-937d-9af2cc7c929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:39.376284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:39.410001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 58/99] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:39.445192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23254f6e-9a44-4dc7-b808-a20ae493dad3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:39.584318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23254f6e-9a44-4dc7-b808-a20ae493dad3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:39.627505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:39.628373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.628660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.628942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.653958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:39.662272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:39.663163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:39.663975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 42/100] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:00:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:39.673765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a41-4797-4e7d-bdb5-afbffb66b182 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.674160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a41-4797-4e7d-bdb5-afbffb66b182 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.674407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f83a41-4797-4e7d-bdb5-afbffb66b182 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.675833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 59/101] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:39.677355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:39.694716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.695009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.695281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.732539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:39.733777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:39.804462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:39.872141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 60/102] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 12:00:39.910845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b78f82-76aa-4d7b-bd03-41efd2206bda None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:39.913854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:39.957561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:39.958248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.958530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.958808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.959611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:39.959905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:39.961315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675e744-97da-4200-9698-63f956d862f6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:39.970757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 43/103] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:00:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:39.973892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fb86-1e2c-45af-be69-96f5761101db None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:39.973892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fb86-1e2c-45af-be69-96f5761101db None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:39.973892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fb86-1e2c-45af-be69-96f5761101db None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:39.973892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 44/104] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:39.976946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04fb86-1e2c-45af-be69-96f5761101db None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.110659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c626-11fe-4afb-8a99-63fe5599d087 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.124950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.163380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.164633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.165107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.165590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.178079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-78dc167c-b312-4688-a303-59aea7902ba4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:40.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:40.187753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:40.188482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:40.191461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 61/105] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:40.193178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c0da-f5ff-4e2f-9ea5-e596dc3bc70f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.193811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c0da-f5ff-4e2f-9ea5-e596dc3bc70f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.194286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c0da-f5ff-4e2f-9ea5-e596dc3bc70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.214217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2dd2c0da-f5ff-4e2f-9ea5-e596dc3bc70f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:40.216531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2dd2c0da-f5ff-4e2f-9ea5-e596dc3bc70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:40.232409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04fb86-1e2c-45af-be69-96f5761101db None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.237453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.295581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.297123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.297627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.298120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.300309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.302929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.304041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 62/106] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:40.305973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de41a-59e6-4958-a748-6f7d7798aa2e None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.306527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de41a-59e6-4958-a748-6f7d7798aa2e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.306632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 45/107] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:39 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:40.307091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de41a-59e6-4958-a748-6f7d7798aa2e None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.308336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.309360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 63/108] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:40.311986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4de41a-59e6-4958-a748-6f7d7798aa2e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.451097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d83f3-d237-4f34-94e9-5e1670c6bf29 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.480237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.484361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4de41a-59e6-4958-a748-6f7d7798aa2e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.501224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.502248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.502976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.504289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.505527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:40.506016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.508422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a5b8b-b469-4ff7-b89b-72e5eb205a25 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.522179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 46/109] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:40.524531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.525061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.525737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.529553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.530522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.530875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.531243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.533048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.535907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.538152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 64/110] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:40 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:40.544552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.552052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:40.553293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:40.592760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.648203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 47/111] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:40.651651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.681812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a6cd-3181-4822-9b1e-daf244c66451 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.723452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.724702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.725272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.725752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.727233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.729082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.731026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 65/112] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:40.732090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.771642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.781086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f2ecf-b2da-4538-98d5-eda1e1fb0ad5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.817737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.818629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.818906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.819166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] PATH_INFO: `/v3/users/2d76bd38c09c4643925333979deda4a4` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.833991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] RBAC: Authorizing `identity:get_user(user_id=2d76bd38c09c4643925333979deda4a4)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.835514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.837681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 48/113] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:40 2022] GET /identity/v3/users/2d76bd38c09c4643925333979deda4a4 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:40.838495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.848918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab06161-6d75-47f7-bf22-3f4cf37aa620 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:40.866902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:40.901469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:40.902515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:40.902882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:40.903301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/a08f17ad10864a879676643d5994d963/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:40.905880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, user_id=a08f17ad10864a879676643d5994d963, role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:40.907574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:40.923291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 66/114] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:40 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/a08f17ad10864a879676643d5994d963/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:40.929145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:40.968195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7be97-6685-4264-b16e-979a6b970b37 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:41.025238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:41.026443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.026868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.027294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.045013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:41.052399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-9e25c548-49a9-4d56-9201-5e0503b807f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:41.058452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:41.059582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:41.060328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:41.061444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 49/115] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:41.063761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fb9a2-f166-401a-8271-29eb56502ca3 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.064345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fb9a2-f166-401a-8271-29eb56502ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.064875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3fb9a2-f166-401a-8271-29eb56502ca3 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.065841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2120d8-3874-446a-a197-83312fa0a02a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:41.067404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 50/116] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:41.072585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fb9a2-f166-401a-8271-29eb56502ca3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:41.107417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:41.108342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.108754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.109164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.110106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:41.111870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90855123-79cb-4c5b-805d-03bb8fef2ac2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:41.158815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 67/117] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:00:40 2022] POST /identity/v3/projects => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:41.161818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2742fc-e9e2-416c-adba-75c9a46b3b83 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.162498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2742fc-e9e2-416c-adba-75c9a46b3b83 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.163020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2742fc-e9e2-416c-adba-75c9a46b3b83 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.165433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 68/118] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:41.166794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2742fc-e9e2-416c-adba-75c9a46b3b83 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:41.206828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:41.251019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3fb9a2-f166-401a-8271-29eb56502ca3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:41.326123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:41.327096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.327695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.328232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.371750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2742fc-e9e2-416c-adba-75c9a46b3b83 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:41.434493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:41.435535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.436067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.436597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.437966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:41.438536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:41.441090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee268e-5214-4e59-94dd-f26d10ef8d04 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:41.455812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 69/119] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:41.458113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9be-8e64-4e49-8a9f-a90bdded0bb2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.458663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9be-8e64-4e49-8a9f-a90bdded0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.459199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29e9be-8e64-4e49-8a9f-a90bdded0bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.467646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:41.471974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:41.485050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ac29e9be-8e64-4e49-8a9f-a90bdded0bb2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:41.487021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ac29e9be-8e64-4e49-8a9f-a90bdded0bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:41.492303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:41.607634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 70/120] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:41.607634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.607634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.607634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.653185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:41.655753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:41.661898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:41.700297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 51/121] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:41.702305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:41.788054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 71/122] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:41.789707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:41.897559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58046d0e-a558-467e-bd6b-952d477cc33c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:41.935186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:41.968264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:41.971690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:41.972162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:41.972577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:41.975300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:41.977003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:41.978887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 52/123] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:41 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:41.981184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.013394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763088a-7ca8-4ff3-9cce-9fba4ef82e4e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.071929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:42.080070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.081241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.081725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.082221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.096965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:42.105607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:42.106527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:42.107263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:42.109966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 72/124] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:42.111232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.178229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66dcc2-531c-4b8a-897d-bba77edfe595 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.237075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.238505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.238960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.239426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.241616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:42.244227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:42.247779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 53/125] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:41 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:42.248997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.277325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf05c80-b37d-4ddc-80ad-60f095c349eb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.309213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:42.319952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.321336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.321838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.322345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.335385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:42.345209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:42.346097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:42.346846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:42.349260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 73/126] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:42.350535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.431067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f96110-2e3f-4582-9a8c-67915dcc5fbb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.459982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:42.473354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5372aa3-3866-4c68-a5b2-6621a48297cd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.489883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.491306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.491813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.492306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.494479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:42.497356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:42.500899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 54/127] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:42.502034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.517858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.518945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.519429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.519932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.520932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:42.521467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:42.523242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:42.533743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 74/128] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:42.535140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.594939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:42.656041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e9bd2-f2fd-4ecc-a913-73f284bcd873 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.692266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58946e-3bec-477c-8fbd-eb475495590d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.718635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.719832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.720776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.720986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.722389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:42.722926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:42.725817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:42.738095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:42.739841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:42.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:42.740926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:42.741606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 75/129] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:42.742361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=2d76bd38c09c4643925333979deda4a4) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:42.743019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:42.743119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.745063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:42.759668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 55/130] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/role_assignments?user.id=2d76bd38c09c4643925333979deda4a4&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:42.761120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:42.861071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:42.949112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267aefc-3c3d-4036-ab48-c9178d2c8a72 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.957710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a5a90-508e-4a7e-afaf-cb4d24897b5b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:42.978586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:43.025234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.028258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.029090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.029928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.030897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.032041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.032395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.032750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/a08f17ad10864a879676643d5994d963/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.033490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.036730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.039919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, user_id=a08f17ad10864a879676643d5994d963, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.040909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 56/131] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:42 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:43.041639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.042832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.072747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 76/132] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:42 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/a08f17ad10864a879676643d5994d963/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:43.073678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.363129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571eed0-fffd-4966-b2d6-347d39bbf0c8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:43.393762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:43.415199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.424378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.424792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.424953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.425790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da809171-58c8-40e9-a4bc-4e082f682555 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:43.426420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.427987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9eb3d-662a-4e04-89af-216585ab4347 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.430626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 57/133] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:43 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:43.432681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881e5b-afbb-4bf5-a59e-6d34ee145ee2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.433563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881e5b-afbb-4bf5-a59e-6d34ee145ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.433563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88881e5b-afbb-4bf5-a59e-6d34ee145ee2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.436153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 58/134] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:43.439213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88881e5b-afbb-4bf5-a59e-6d34ee145ee2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.505533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.507186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.507688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.507979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] PATH_INFO: `/v3/projects/ba6c02055d0b42aab693c168e4f2d58b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.510482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] RBAC: Authorizing `identity:get_project(project_id=ba6c02055d0b42aab693c168e4f2d58b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.512839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f192e-2ffb-4bbc-95f1-8bcc4b0b1621 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.515584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 77/135] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:43 2022] GET /identity/v3/projects/ba6c02055d0b42aab693c168e4f2d58b => generated 312 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:43.532172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.532589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.532899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.571116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:43.573206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:43.608664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88881e5b-afbb-4bf5-a59e-6d34ee145ee2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:43.637349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:43.678431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 78/136] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:43.679398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.713260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.714219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.714498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.714772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.719803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.721292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.723773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 59/137] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:43 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:43.729298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.805423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:43.897808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a405-bce4-4e82-99fc-0a469d2bd6ff None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:43.906746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa33d9d-db06-4416-94e8-72f6acb4cfc6 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:43.949261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.949261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.949524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.949624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] PATH_INFO: `/v3/projects/ba6c02055d0b42aab693c168e4f2d58b/users/a08f17ad10864a879676643d5994d963/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:43.951692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6c02055d0b42aab693c168e4f2d58b, user_id=a08f17ad10864a879676643d5994d963, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:43.953226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:43.970979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 79/138] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:43 2022] PUT /identity/v3/projects/ba6c02055d0b42aab693c168e4f2d58b/users/a08f17ad10864a879676643d5994d963/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:43.971863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:43.992266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:43.993525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:43.994014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:43.994504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.034271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:44.035862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:44.049854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:44.050758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:44.051528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:44.054029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 60/139] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:00:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:44.064604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:44.144220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d8b6c-59bb-454b-971e-4c7ec9b9a56e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:44.281031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:44.281967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.282245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.282519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/7aeac511a0044d6781fee6120328a71c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.287062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, user_id=7aeac511a0044d6781fee6120328a71c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:44.288650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff14da2-2a89-4b04-8bb0-e4ac981aef86 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:44.310505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 80/140] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:43 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/7aeac511a0044d6781fee6120328a71c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:44.337154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10378202-7168-4c4a-b628-650291cd3c33 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.337541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10378202-7168-4c4a-b628-650291cd3c33 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.337829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10378202-7168-4c4a-b628-650291cd3c33 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.339934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 81/141] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:00:44.344641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc2e91-c4ed-40af-a0b8-646ac94e91a5 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:44.350228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.350642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.350905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.366137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:44.384060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:44.385404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:44.401927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:44.402661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.402950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.403238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.404105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:44.404424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:44.410693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:44.427845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 61/142] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:44.430013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:44.558363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 82/143] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:44.594078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:44.645779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:44.723426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee200e-e183-4175-ac46-d9b562bb3ca3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:44.827073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:44.827584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425cbe6-bfb3-4b43-884f-6a7c20ee845c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:44.828025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.828318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.828603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.833798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:44.835351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:44.838097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 62/144] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:44 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:44.853273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:44.879602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:44.880435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:44.880721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:44.880998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:44.884505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:44.886114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:44.892320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 83/145] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:44.898105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:44.988130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:45.062778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:45.073254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970101a-b8d1-4469-965d-52f633b1fb8c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:45.152170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d53e-d647-46c8-b57e-5d0484eb75fd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:45.163531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:45.163531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:45.163531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:45.163531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:45.164073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:45.165368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:45.223685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 84/146] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:44 2022] POST /identity/v3/users => generated 283 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:45.263100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:45.263704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:45.263992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:45.264273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:45.375620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:45.377144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:45.442895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:45.534964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 63/147] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:45.541299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:45.702840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3ce53-a48c-4629-9d81-41720f909f75 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:45.746112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:45.747029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:45.747653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:45.747945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:45.749404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:45.750969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74d9a5-6149-4c4f-beed-216ffe8ca7bf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:45.758187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 85/148] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:45 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:45.759932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:45.937785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d8595-13f2-413b-85bb-c2f9544cb992 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:45.984068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:45.984810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:45.985097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:45.985375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:45.986244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:45.986559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:45.988150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bfbbc-5d8c-4297-bf0d-1bb09c630900 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:45.995527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 64/149] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:45 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:46.384599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab630d7-2c85-457a-b5f4-bfd836c9538e None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:46.384816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab630d7-2c85-457a-b5f4-bfd836c9538e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:46.385159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab630d7-2c85-457a-b5f4-bfd836c9538e None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:46.387024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 86/150] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:46.392574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:46.393130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:46.393589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:46.413559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:46.415153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:46.418131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:46.503117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 65/151] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:46.527745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab630d7-2c85-457a-b5f4-bfd836c9538e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:46.544579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:46.668570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab630d7-2c85-457a-b5f4-bfd836c9538e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:46.721491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:46.722333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:46.722616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:46.722898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:46.739524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:46.747706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:46.748649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:46.749377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 87/152] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:00:46.755135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:46.880095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf5a7-1a74-48a8-8d90-e220e59a9bf4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:46.932660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:46.933268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:46.933557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:46.933852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:46.934672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:46.934992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:46.936477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f4a8d-4ea9-47c8-983a-39b3f787834b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:46.948556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:46.952953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 66/153] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:46.961302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:47.024884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaffae8-cba7-480a-85b2-513f95e56e43 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.025297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaffae8-cba7-480a-85b2-513f95e56e43 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.025593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaffae8-cba7-480a-85b2-513f95e56e43 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.029344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 67/154] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:47.037307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30786197-4cc6-4a86-8160-6d3c8872ccf8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.037926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30786197-4cc6-4a86-8160-6d3c8872ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.038460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30786197-4cc6-4a86-8160-6d3c8872ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.062906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-30786197-4cc6-4a86-8160-6d3c8872ccf8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:47.064557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-30786197-4cc6-4a86-8160-6d3c8872ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:47.213825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:47.253607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 68/155] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:47.276901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059d21-85a0-4922-878f-1d864914cff7 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:47.287884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23618-2e4a-4610-bdb9-813cad450ea3 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.288522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23618-2e4a-4610-bdb9-813cad450ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.288995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23618-2e4a-4610-bdb9-813cad450ea3 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.290635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 69/156] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:00:47.296594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b174ee-1d17-415c-a92d-cd5b44402f29 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.297207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b174ee-1d17-415c-a92d-cd5b44402f29 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.297677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b174ee-1d17-415c-a92d-cd5b44402f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.330829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-76b174ee-1d17-415c-a92d-cd5b44402f29 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:47.332115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-76b174ee-1d17-415c-a92d-cd5b44402f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:47.345044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:47.346656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.347051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.347561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] PATH_INFO: `/v3/users/2d76bd38c09c4643925333979deda4a4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.349798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] RBAC: Authorizing `identity:get_user(user_id=2d76bd38c09c4643925333979deda4a4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:47.352597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:47.356136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 88/157] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:46 2022] GET /identity/v3/users/2d76bd38c09c4643925333979deda4a4 => generated 289 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:47.357612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:47.485518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 70/158] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:47.493608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78504bd2-c021-4b29-9a44-16f6feef45c9 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.494134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78504bd2-c021-4b29-9a44-16f6feef45c9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.494564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78504bd2-c021-4b29-9a44-16f6feef45c9 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.496861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 71/159] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:47.497796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78504bd2-c021-4b29-9a44-16f6feef45c9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:47.610365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89dca-1165-423f-8e41-57787d9cbb06 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:47.632830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:47.688815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:47.689649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.689933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.690212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.716322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:47.726174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:47.732476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78504bd2-c021-4b29-9a44-16f6feef45c9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-eaa6424c-d3ee-4fd3-b817-b9ca26fe3447 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:47.734355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:47.735173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:47.736053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:47.736646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 89/160] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:47.738075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.738075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.738244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.789286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:47.790852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:47.839270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:47.840201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:47.840487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:47.840764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:47.843242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:47.844773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:47.849499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 72/161] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:47 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:47.853660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:47.974246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8886-c943-45bd-b33e-c4be63892148 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.002123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 90/162] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:48.002927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.007540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:48.028004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.028832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.029111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.029381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.050752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:48.058505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:48.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:48.060201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:48.062127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 73/163] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:48.068057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.204518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd799e0-bc08-4997-adc8-daadbd6c055f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.236915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866cade-46c5-4510-8d21-69c7f9495b0a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.277429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.278715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.279193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.279686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/2d76bd38c09c4643925333979deda4a4/roles/8cf34c361caf421f8f85b4274ca9ac1a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.283128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, user_id=2d76bd38c09c4643925333979deda4a4, role_id=8cf34c361caf421f8f85b4274ca9ac1a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:48.285109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:48.298452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:48.310637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 74/164] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:48 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/users/2d76bd38c09c4643925333979deda4a4/roles/8cf34c361caf421f8f85b4274ca9ac1a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:48.313099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.315831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.316912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.318043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.319689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.320934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:48.322453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:48.324347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:48.347329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 91/165] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:48.350946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.411984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:48.475291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cee00b-64a4-474e-9015-4b28fdcd96dd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.525151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.526079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.526368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.526640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.551964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:48.560453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:48.561352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:48.562585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 75/166] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:48.563216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.580398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7573c18-b8a8-4117-8095-1cb0554aa0fe None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.623254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.623986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.624268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.624543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.625339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:48.628062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:48.629636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:48.646620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 92/167] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:48.651804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.719405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:48.772571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92946aba-c381-4951-a664-9270bcdab8a2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.814211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.814826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.815109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.815395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.816194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:48.816500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:48.817968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:48.826009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be2ce2-1ffb-4364-a2ce-87420340b54e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:48.831718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 93/168] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:48.832719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:48.839983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:48.890645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:48.891573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:48.891860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:48.892124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:48.894303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:48.898298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:48.900455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 76/169] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:48.901550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:49.058882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761090c-2104-4abe-aa0a-dcc77a6469a9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:49.092309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:49.121527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:49.122445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:49.122726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:49.122998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] PATH_INFO: `/v3/users/7aeac511a0044d6781fee6120328a71c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:49.124368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] RBAC: Authorizing `identity:get_user(user_id=7aeac511a0044d6781fee6120328a71c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:49.127042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:49.129358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 94/170] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/users/7aeac511a0044d6781fee6120328a71c => generated 247 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:49.130689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:49.144955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e641898-b077-4aec-a183-fc4e47d9f5f9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:49.266643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:49.267578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:49.267857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:49.268127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] PATH_INFO: `/v3/users/a08f17ad10864a879676643d5994d963` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:49.269467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] RBAC: Authorizing `identity:get_user(user_id=a08f17ad10864a879676643d5994d963)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:49.271007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:49.272972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 77/171] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:48 2022] GET /identity/v3/users/a08f17ad10864a879676643d5994d963 => generated 275 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:49.273547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:49.363086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:49.383274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e84f6-0e08-40b3-8263-a52ba278e542 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:49.427640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d2c9c-768c-4950-bbfe-6e26484947fa None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:49.483977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:49.484664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:49.484943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:49.485217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:49.491345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:49.492057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:49.492336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:49.492604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:49.593925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:49.612682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:49.614280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc23a-bc3d-44bc-b07b-c3bb5a5959e9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:49.655274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:49.656800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:49.766248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:49.851792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 95/172] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:49.853004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfa89b-fd5c-4d57-beab-1c09ecaa1b02 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:49.867922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfa89b-fd5c-4d57-beab-1c09ecaa1b02 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:49.868211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfa89b-fd5c-4d57-beab-1c09ecaa1b02 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:49.869719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 96/173] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:49 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:49.870619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfa89b-fd5c-4d57-beab-1c09ecaa1b02 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:49.900925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 78/174] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:49.901619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.026765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28487d5b-bb8a-4322-a077-032e384d039e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.072210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.078197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.079921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.080208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.080482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.082423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.084762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93cd67-296c-465a-a54e-91ba97717753 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.091777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 79/175] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:49 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:50.091777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.091777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.091777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.100976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfa89b-fd5c-4d57-beab-1c09ecaa1b02 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.109224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:50.110555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:50.119621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.144483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.145625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.146111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.146704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.214307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 80/176] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:50.216893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.266839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.269159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295b226-6063-4b52-a7eb-fc3934b8dbf5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.355634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf165d1-cb64-4b3e-a0db-a326a86ceb5d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.363632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.384732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 97/177] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:50.387383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046008-fe0f-4469-8541-922591658e5d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.388222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046008-fe0f-4469-8541-922591658e5d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.388677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8046008-fe0f-4469-8541-922591658e5d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.391219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 98/178] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:50.392304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8046008-fe0f-4469-8541-922591658e5d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.400425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.401968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.402259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.402536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.404323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.406158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.408061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 81/179] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:50.412731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.427320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.511420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8046008-fe0f-4469-8541-922591658e5d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.528429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8104d-2cad-4ce5-94c0-7a22c07dc2f1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.550068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.550746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.551024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.551296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.552179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:50.552498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.554112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68390-30a3-478f-a8cc-f705b4c4117d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 99/180] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c279-c7ec-449a-9a7a-3804968ad49a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c279-c7ec-449a-9a7a-3804968ad49a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c279-c7ec-449a-9a7a-3804968ad49a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 100/181] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:50.566939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c279-c7ec-449a-9a7a-3804968ad49a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.579802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.580644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.580924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.581199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.594860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-9c4cfaeb-cba1-45ec-b5b3-f98dc4faa8b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:50.602944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:50.604091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:50.604981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 82/182] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:50.605997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.605997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.606239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.625101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:50.626390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:50.639583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.716397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c279-c7ec-449a-9a7a-3804968ad49a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.724391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 83/183] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:50.728657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.731920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:50.779391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.780319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.780596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.780865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] PATH_INFO: `/v3/projects/ba6c02055d0b42aab693c168e4f2d58b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.782135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] RBAC: Authorizing `identity:get_project(project_id=ba6c02055d0b42aab693c168e4f2d58b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.783774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6795d25-cb21-46fe-a7a6-95a8f29ccfba None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.785887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 101/184] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/projects/ba6c02055d0b42aab693c168e4f2d58b => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:50.787286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.787639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.787995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.808688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:50.810253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:50.884022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd67e45-e396-4d92-847d-4344f97123f5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:50.897352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 102/185] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:50.898361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.934840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:50.935710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:50.936024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:50.936306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:50.937169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=7aeac511a0044d6781fee6120328a71c) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:50.937484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:50.939069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:50.946035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 84/186] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/role_assignments?user.id=7aeac511a0044d6781fee6120328a71c&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:50.946710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:50.958233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.022336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af907ee-4dba-4f55-9571-89253e848e16 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.046586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.065879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.066976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.067523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.068092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.069096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:51.069745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.071424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.081985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 103/187] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.088711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.099094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582068c-7f68-41a4-a414-fa77a51c36df None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.141848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.143050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.143679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.144211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.164700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:51.173697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:51.174631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:51.175617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 85/188] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:00:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:51.176349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.232627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821302c-6932-4049-8e6c-411d370552d2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.246522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.275766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.276527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.276814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.277127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.278016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=ba6c02055d0b42aab693c168e4f2d58b, user.id=a08f17ad10864a879676643d5994d963) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:51.278334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.279864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.290027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 104/189] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/role_assignments?user.id=a08f17ad10864a879676643d5994d963&scope.project.id=ba6c02055d0b42aab693c168e4f2d58b&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.291731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.321311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92b039-3dc6-4f95-85af-3f0fec1cb33f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.368007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.388370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.389943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.390513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.391063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.393491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.396375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.404480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 86/190] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.404480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.431465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217e38e-8b44-4226-a306-93d73a51ca06 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.480345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.481631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.482899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.483391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.509078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.510894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.514166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 105/191] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.517151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.572295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.582841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d54be-f116-4c5c-9898-bb2b4d2bc3f2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.644887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.646009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.646445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.646882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.648325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:51.648877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.651533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.652100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b8b8-d200-4a20-8490-ac7ac6c0bbc4 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.666284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 87/192] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.667875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.690038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.690862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.691139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.691414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.692211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.693686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7563d-6a39-4b84-b286-912e5b79fb7f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.716035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 106/193] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:51 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:51.732005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e2d-65fb-4dad-a5f6-177dc68b46d0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.732005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e2d-65fb-4dad-a5f6-177dc68b46d0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.732005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e2d-65fb-4dad-a5f6-177dc68b46d0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.732005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 107/194] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:51.732005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b4e2d-65fb-4dad-a5f6-177dc68b46d0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:51.754533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.815118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4e04-6813-4120-b1c8-63c420c78722 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.844513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:51.865133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b4e2d-65fb-4dad-a5f6-177dc68b46d0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:51.880428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.881522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.881934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.882375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.903181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:51.904144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.904559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.904842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.906235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.907813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cbe60-5555-40f9-b8b3-5277c56d4ec6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:51.909594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 108/195] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:51.916262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:51.916668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:51.916904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:51.935023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:51.936336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:51.993372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:51.995007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:52.040768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 109/196] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:52.042324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.077346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.158806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 88/197] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:52.160513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.216384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.235397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc628b-3826-4c6c-a982-8b984b288bb4 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.273418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.273970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.274365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.274704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:52.340371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc126c9-1a90-4953-95bb-b5cb8f37f32c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.349553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:52.351064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:52.409283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.411239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.412007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.412748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:52.435197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:52.441694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:52.455297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:52.456997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:52.458421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:52.460191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 110/198] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:52.461300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.463246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 89/199] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:52.474857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.499467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.591238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220fa0-6771-4e3b-b901-bd8bedb39732 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.636538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.637818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.638456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.639027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:52.640330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:52.642097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:52.649243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 111/200] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:52.650244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.666771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ccbc5-76a9-4be4-bbfd-9d04917637e4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.723535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.741834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.743188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.743728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.744215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:52.746558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:52.749256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:52.752868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 90/201] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:52.754249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.773484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df80a9a-ef90-4197-b60a-f62127069057 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.815434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.816505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.816788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.817060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:52.818873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:52.819185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:52.820751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:52.847498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 112/202] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:52.849384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:52.902364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:52.934279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd900-def6-4384-9594-60f76f1715d7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:52.997120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:52.998055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:52.998517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:52.998981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:53.000652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=cf278e4c9a5a4b94b797ff20f395d5c4) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:53.001166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:53.003748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:53.014531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 91/203] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/role_assignments?user.id=cf278e4c9a5a4b94b797ff20f395d5c4&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:53.015953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:53.041494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5a2f1-2446-4529-b926-3d623c09f5f2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:53.090291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:53.129072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:53.130278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:53.130803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:53.131327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:53.133198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=cf278e4c9a5a4b94b797ff20f395d5c4) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:53.133818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:53.136879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:53.148138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 113/204] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:52 2022] GET /identity/v3/role_assignments?user.id=cf278e4c9a5a4b94b797ff20f395d5c4&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:53.219406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eab0-3ba9-4078-8508-a5372fd5760c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:53.288541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:53.290512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:53.291279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:53.291781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] PATH_INFO: `/v3/users/2d76bd38c09c4643925333979deda4a4` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:53.294387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] RBAC: Authorizing `identity:get_user(user_id=2d76bd38c09c4643925333979deda4a4)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:53.297522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:53.301022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 92/205] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:53 2022] GET /identity/v3/users/2d76bd38c09c4643925333979deda4a4 => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:53.307740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:53.507083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b90b4b-4aa7-4eb8-9280-55fceb7f96ff None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:53.595815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:53.596938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:53.597504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:53.597999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:53.700970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:53.732691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:53.734203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:53.876210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 114/206] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:53.882098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:53.929873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:54.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105a889-4429-493d-ac0d-7cd9ea8f872f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:54.132626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:54.134055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:54.134545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:54.135029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:54.137075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:54.139874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f72fa-8e4a-4e8c-a335-e8906bee1571 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:54.143220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 93/207] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:53 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:54.149999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:54.301814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:54.339960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf60d3a-b738-490b-b1d0-2ad260992c66 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:54.408383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:54.409499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:54.409978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:54.410469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:54.412087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, scope.project.id=43f8b4157afc45d28ef515833ee374d1, user.id=2d76bd38c09c4643925333979deda4a4) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:54.412518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:54.415024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acea58-f9b7-4825-b62a-4b419b280b37 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:54.426362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 115/208] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:54 2022] GET /identity/v3/role_assignments?user.id=2d76bd38c09c4643925333979deda4a4&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:54.806322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa800a-fe19-4730-af81-4a60b8a5a219 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:54.806496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa800a-fe19-4730-af81-4a60b8a5a219 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:54.806785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa800a-fe19-4730-af81-4a60b8a5a219 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:54.808689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 94/209] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:54.814760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:54.815060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:54.815318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:54.840689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:54.841928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:54.927688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 116/210] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:54.955691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa800a-fe19-4730-af81-4a60b8a5a219 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.073849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:55.102462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa800a-fe19-4730-af81-4a60b8a5a219 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:55.140419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:55.141245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.141531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.141809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.143100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:55.146510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9e564-e5a3-4929-bc6c-77e21a4f0790 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:55.146510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 95/211] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:55.159652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.242175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b9e1-a93b-4fee-b21d-e14db4fbd374 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.242497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b9e1-a93b-4fee-b21d-e14db4fbd374 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.242728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8b9e1-a93b-4fee-b21d-e14db4fbd374 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.244239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 96/212] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:00:55.247962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:55.250232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.250516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.250795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.269562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:55.270798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:55.290071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8576c0-dfe1-4bf6-a41f-8a6b2b228472 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:55.342997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:55.343806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.344359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.344635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.346061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:55.347928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b84eead-57b1-4496-ab20-c941bc31f23f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:55.355899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 97/213] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:55.381272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.384790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 117/214] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:55 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:55.493279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b85a3e-3610-47b9-a811-a35dc6311c65 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.493643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b85a3e-3610-47b9-a811-a35dc6311c65 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.493834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b85a3e-3610-47b9-a811-a35dc6311c65 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.495706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 118/215] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:55.499704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:55.503058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.503753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.504235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.526267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:55.528613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:55.529605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62619-af5e-401b-bf02-ebc713905d03 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:55.597858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:55.599183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.599701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.600029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.621934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:55.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:55.638460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:00:55.639218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:55.640587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 98/216] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:55.648217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.650771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 119/217] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:55.683395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.759356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:55.800237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c38b8-72c6-48f7-a7ca-4b078b52bae9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:55.843057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb27a1-b781-4128-9f9d-2f1c9a280929 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:55.852983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:55.854463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.855038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.855754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.857363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:55.858104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:55.861155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:55.879905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 99/218] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:55.887610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:55.891006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:55.896217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:55.897297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:55.897726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:55.898144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:55.924552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:55.939569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:55.940633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:00:55.941421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:55.943301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 120/219] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:00:55.953174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.037808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f950a4-1f83-4ebd-89d1-f1fe6b284213 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.085347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.086270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.086922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.087195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] PATH_INFO: `/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.118604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] RBAC: Authorizing `identity:get_user(user_id=70b0bc75a23c4e88ba4274ce5f77fb40)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.120139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.122297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 100/220] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:55 2022] GET /identity/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:56.153917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.154076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26718017-50a2-4813-97ce-8703027a17e8 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.189223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.228637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.229851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.230354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.230841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.232331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:56.232880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.241400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.261233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 121/221] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:56.273238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.283450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.367201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf10d93-d8bd-4034-af6f-446fa0d62231 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.426550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.427245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.427577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.427860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.473168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6119c-4892-42d9-8ea6-5b0aa1db6895 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.506712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.508281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06abbb-7492-48c3-ace1-680703c70b06 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.512217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.513172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.513461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.513738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.515037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.516600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c0473-aa02-454d-9b06-b7cb609044fa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.518707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 122/222] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:56.520115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83947f7-69ac-4b8e-8ad7-e004e784e110 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.520408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83947f7-69ac-4b8e-8ad7-e004e784e110 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.520685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83947f7-69ac-4b8e-8ad7-e004e784e110 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.526707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 123/223] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:56.527763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83947f7-69ac-4b8e-8ad7-e004e784e110 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.599907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.612547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.621408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 101/224] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:00:56.623100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.623471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.623846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.643050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:56.644492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:56.658846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83947f7-69ac-4b8e-8ad7-e004e784e110 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.701005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.701871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.702207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.702486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.703924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.705618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.707643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 124/225] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:56.708469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.729177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 102/226] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:00:56.730262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.842830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba72b2-1c2c-47b6-a9e1-511401414732 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.846384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39c2b-adb9-42ec-8a3f-e5ab8ab923e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:56.865150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.883731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.884988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.885344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:56.885803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.886225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/cf278e4c9a5a4b94b797ff20f395d5c4/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.886542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:56.886916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:56.887320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:56.888696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.889139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, user_id=cf278e4c9a5a4b94b797ff20f395d5c4, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:56.890377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.890917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:56.892779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 125/227] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:56.893776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:56.897532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:56.913543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 103/228] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:56 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/cf278e4c9a5a4b94b797ff20f395d5c4/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:56.920928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:57.009008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c9e2c-6706-4ec1-bed6-31a1261cbb1d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:57.048075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:57.048913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:57.049194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:57.049472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:57.061891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:57.074575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:57.075550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:00:57.076324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 126/229] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:57.083067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:57.151870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:57.225495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9faf3-8d71-498b-97a7-053fa54c235c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:57.303662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:57.305020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:57.305521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:57.306094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:57.307752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:57.307872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0f56b-f887-49ac-b88d-067f2a44a060 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:57.308380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:57.311433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:57.340144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 127/230] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:00:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:57.351772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:57.362660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:57.363469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:57.363867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:57.364214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:57.365160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=70b0bc75a23c4e88ba4274ce5f77fb40) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:57.365594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:57.367182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:57.374949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 104/231] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:00:56 2022] GET /identity/v3/role_assignments?user.id=70b0bc75a23c4e88ba4274ce5f77fb40&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 259 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:57.562737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56316f86-dd4e-4fef-8d34-f6173bb5c467 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:57.577786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:57.629686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:57.631272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:57.631813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:57.632312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:57.634583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:57.637604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:57.640984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 128/232] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:57 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:57.649302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:57.682698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:57.809544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfecf96-0d37-4ee0-a684-032d7c8c4c5b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:57.862597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:57.863396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:57.863710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:57.863990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:57.865243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:57.866741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea6a53-f3fb-4996-b140-9ce1a216ea0c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:57.868834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 105/233] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:00:57 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:00:57.878628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:58.096697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324054-52e1-4b91-bd96-11ba3e1da90b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:58.181968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:58.183644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.184119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.184595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/cf278e4c9a5a4b94b797ff20f395d5c4/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.190261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, user_id=cf278e4c9a5a4b94b797ff20f395d5c4, role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:58.192689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff6f51-22f1-4977-8f74-0daaa11f91fe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:58.232673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 129/234] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:00:57 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/cf278e4c9a5a4b94b797ff20f395d5c4/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:00:58.473655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a702edb-07e1-4139-9b94-e128d964847b None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.474415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a702edb-07e1-4139-9b94-e128d964847b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.474415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a702edb-07e1-4139-9b94-e128d964847b None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.475817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 106/235] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:00:58.482318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e810-a5fc-4842-8d49-d3bdfaa70de1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.482763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e810-a5fc-4842-8d49-d3bdfaa70de1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.483144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308e810-a5fc-4842-8d49-d3bdfaa70de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.506665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d308e810-a5fc-4842-8d49-d3bdfaa70de1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:58.507976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d308e810-a5fc-4842-8d49-d3bdfaa70de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:58.560668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:58.614587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 130/236] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:58.639069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a702edb-07e1-4139-9b94-e128d964847b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:58.761288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:58.786218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a702edb-07e1-4139-9b94-e128d964847b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:58.809076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2651f-d1b9-4a02-8001-85c426750cd5 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.809515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2651f-d1b9-4a02-8001-85c426750cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.809945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2651f-d1b9-4a02-8001-85c426750cd5 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.812062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 131/237] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:00:58.818527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410665ae-94d2-4669-a618-724c954b53dd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.819123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410665ae-94d2-4669-a618-724c954b53dd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.819700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410665ae-94d2-4669-a618-724c954b53dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.846037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-410665ae-94d2-4669-a618-724c954b53dd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:00:58.848297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-410665ae-94d2-4669-a618-724c954b53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:00:58.865614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:58.866961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:58.867392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:58.870689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:58.893844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:00:58.905478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:00:58.907102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:00:58.908484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 107/238] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:00:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:00:58.916701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:59.068682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20974b8-4146-4c70-8770-91638997d400 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:59.086752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 132/239] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:00:59.130925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:59.131681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:59.131970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:59.132251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:59.133055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:00:59.133368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:59.134817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:59.150735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410665ae-94d2-4669-a618-724c954b53dd None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:59.159628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 108/240] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:00:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:59.173599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:59.181937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:59.330172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:59.338576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410665ae-94d2-4669-a618-724c954b53dd None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:59.383772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:59.384758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:59.385050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:59.385382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:59.386785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:59.388311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:59.390328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 133/241] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:00:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:59.398645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:59.435714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86516faa-2887-40d5-bf6f-71a074f4beb1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:59.481779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:59.482930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:59.482930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:59.483386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] PATH_INFO: `/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:59.499691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] RBAC: Authorizing `identity:get_user(user_id=ad5b9dc286cf4d0fb42c1a91f9867369)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:59.501233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:59.503585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 109/242] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:00:59 2022] GET /identity/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369 => generated 297 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:00:59.509306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:00:59.606949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8accbf-9566-4aa9-9d1f-24672caa8bfd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:59.610499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:00:59.648472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:59.649179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:59.649448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:59.649728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:59.650483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:59.652064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:00:59.683992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 134/243] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:00:59 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:00:59.750898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b23157-cc30-4d43-9c4c-93b037ac193d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:00:59.813831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:00:59.814940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:00:59.815434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:00:59.815959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:00:59.928722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:00:59.930264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:00.004470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:00.056046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 110/244] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:00.061888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:00.084393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:00.199998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f0a08-2fe9-4b76-a19b-30c80525bf8a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:00.237629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:00.238616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.238914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.239186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.240490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:00.242007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a6293-fff7-4ad9-9bb4-ab133a047e90 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:00.244060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 135/245] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:00 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:00.250928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:00.474644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01f59d-f0b1-463d-8a1c-5ce326109252 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:00.540787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:00.541994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.542449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.542929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.544702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=ad5b9dc286cf4d0fb42c1a91f9867369) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:00.545266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:00.548059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc9e12-9df6-4894-b9fd-b0bd5f234501 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:00.559721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 111/246] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:00 2022] GET /identity/v3/role_assignments?user.id=ad5b9dc286cf4d0fb42c1a91f9867369&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:00.618291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cb71b-69ed-4e1c-9cfa-b78206fe761e None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.619242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cb71b-69ed-4e1c-9cfa-b78206fe761e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.619242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cb71b-69ed-4e1c-9cfa-b78206fe761e None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.620756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 136/247] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:00.626620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.626994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.627230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.647370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:00.648640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:00.663976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d96c-95e9-44d4-85e4-8a13a24c92d3 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.664448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d96c-95e9-44d4-85e4-8a13a24c92d3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.667019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16d96c-95e9-44d4-85e4-8a13a24c92d3 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.672815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 137/248] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:00.681597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.682077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.682549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.683330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:00.744975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 112/249] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:00.748648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:00.751068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:00.771115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:00.903802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd24ba-f971-40d3-b968-78f8c4788fa5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:00.906392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 138/250] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:00.944486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:00.945353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:00.945637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:00.945905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:00.958181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:00.962290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:00.968201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:00.973484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 113/251] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:00.974991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:01.000156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:01.028522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:01.138036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38094328-3b08-4579-9120-4ac0cf77c998 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:01.264254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:01.265322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:01.265814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:01.266300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:01.267725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:01.268270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:01.277256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:01.305224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 114/252] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:01.314670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:01.337939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba77652-f395-4450-802f-e04cfb080c5f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:01.427529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:01.427529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:01.427529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:01.427529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:01.445595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:01.462172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:01.463935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:01.465431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 139/253] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:01.476622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:01.566437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:01.656536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519b344-5740-49f5-8255-8ee7821f7192 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:01.714810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:01.742530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:01.744154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:01.744600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:01.745112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] PATH_INFO: `/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:01.747914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] RBAC: Authorizing `identity:get_user(user_id=70b0bc75a23c4e88ba4274ce5f77fb40)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:01.750517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:01.755875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 115/254] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:01 2022] GET /identity/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40 => generated 297 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:01.765060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:01.801647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece819a-2ae5-43ec-9b7b-ccbe34715c0b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:01.878240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:01.878853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:01.879143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:01.879425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:01.880257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:01.880573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:01.882056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:01.905431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 140/255] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:01.906406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.048446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b972cea-2199-400c-8ae7-4ff95c7641cf None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.087597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.088756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.089040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.089318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.090619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:02.092151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f715e-0cfc-4096-b636-16d3f3c25213 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:02.094495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 141/256] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:01 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:02.095717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f586bc-5914-42d9-9e0f-5b9490bbf08d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.095996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f586bc-5914-42d9-9e0f-5b9490bbf08d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.096285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f586bc-5914-42d9-9e0f-5b9490bbf08d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.098582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 142/257] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:02.104594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f586bc-5914-42d9-9e0f-5b9490bbf08d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.117949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb66fd-fa93-418e-a717-f50b8e810e49 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.118941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:02.238503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f586bc-5914-42d9-9e0f-5b9490bbf08d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.272283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.273190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.273469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.273748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.275441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.276285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.276633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.276977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.280298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:02.281799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515b90f-6a19-4c2e-87d0-5e6442fb009c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:02.283969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 116/258] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:01 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:02.285401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.285557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.285770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.324257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:02.325508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:02.356771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:02.387537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:02.389047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:02.406864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:02.525591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 117/259] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:02.526784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.535861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 143/260] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:02.543966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.669196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2443b-a450-4c01-a149-f72a2ba1492f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.708523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.709461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.709736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.710012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/70b0bc75a23c4e88ba4274ce5f77fb40/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.712667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, user_id=70b0bc75a23c4e88ba4274ce5f77fb40, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:02.714249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:02.738034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 118/261] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:02 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/70b0bc75a23c4e88ba4274ce5f77fb40/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:02.740280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.748497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:02.755849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08a9be-e13a-4cfd-8080-b9f7415822a3 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.789817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:02.874898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.876420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.877036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.877506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.884546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:02.887249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:02.890648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 144/262] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:02 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:02.892090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c200b7-3d81-492f-a359-babcb53aa5b6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:02.901141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:02.956290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:02.957068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:02.957355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:02.957640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:02.976631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:02.985119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:02.986776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:02.988946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 119/263] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:02.992836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:03.125944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:03.216920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbd63f-e762-4a55-a9ff-62f9f90ec741 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:03.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49f4d6-c482-4f4a-8adf-8a1da81915f5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:03.281049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:03.282210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:03.282698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:03.283192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:03.284616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:03.285156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:03.287838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:03.311060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:03.311270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 120/264] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:03.311791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:03.312078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:03.312353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:03.313230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=cf278e4c9a5a4b94b797ff20f395d5c4) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:03.313405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:03.313575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:03.315029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:03.327030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 145/265] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:02 2022] GET /identity/v3/role_assignments?user.id=cf278e4c9a5a4b94b797ff20f395d5c4&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:03.511218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b984e-cb5e-4966-a2b9-bb03c05061b0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:03.550525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:03.560289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:03.561211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:03.561493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:03.561770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] PATH_INFO: `/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:03.564831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] RBAC: Authorizing `identity:get_user(user_id=cf278e4c9a5a4b94b797ff20f395d5c4)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:03.567023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:03.569115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 121/266] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:03 2022] GET /identity/v3/users/cf278e4c9a5a4b94b797ff20f395d5c4 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:03.578054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:03.631627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:03.709378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbba91-3022-45d6-be50-ec6da900f5bd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:03.760246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:03.761891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:03.762386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:03.762883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:03.869349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:03.871421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:03.923983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:03.997192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 146/267] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:04.005782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.175837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01a2b4-a26f-4cd6-8363-a5b2858e3f12 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:04.232222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:04.233126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.233408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.233685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.240496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:04.242036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f24a7f-0442-47bc-94fa-15991e2f5120 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:04.244190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 122/268] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:04.253392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.418590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeafcd5-1bc9-4ebb-a30b-0f9aa44df01b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:04.421325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558b878-552d-45f2-b0b2-4a34bc3e6e37 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.421615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558b878-552d-45f2-b0b2-4a34bc3e6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.421894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558b878-552d-45f2-b0b2-4a34bc3e6e37 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.423704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 123/269] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:04.436039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e009dd-68ac-48b2-9885-be15be4ef67d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.436359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e009dd-68ac-48b2-9885-be15be4ef67d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.436607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e009dd-68ac-48b2-9885-be15be4ef67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.465824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:04.466518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.466797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.467079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.467978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=cf278e4c9a5a4b94b797ff20f395d5c4) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:04.468292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:04.469832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11ed3-796d-402c-9ff2-ca5632717b88 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:04.472786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-28e009dd-68ac-48b2-9885-be15be4ef67d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:04.474017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-28e009dd-68ac-48b2-9885-be15be4ef67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:04.488349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 147/270] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/role_assignments?user.id=cf278e4c9a5a4b94b797ff20f395d5c4&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:04.489550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5fc62-0f01-4c40-92c0-20e781c5edc6 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.489850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5fc62-0f01-4c40-92c0-20e781c5edc6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.490111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5fc62-0f01-4c40-92c0-20e781c5edc6 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.491526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 148/271] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:04.497456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.497770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.498017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.503292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:04.519025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:04.520381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:04.562697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:04.580268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 124/272] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:04.584150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431efb54-ab7c-4108-8891-f8732ec2bae2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.584449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431efb54-ab7c-4108-8891-f8732ec2bae2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.584717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431efb54-ab7c-4108-8891-f8732ec2bae2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.586699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 125/273] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:04.597452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.597901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.599455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.624053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:04.625292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:04.625426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 149/274] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:04.626330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.733250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 126/275] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:04.735062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.760403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b0613-4f38-4fd4-af20-c0c666e87e8d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:04.804194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:04.812369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:04.816326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.816326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.816326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.834486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:04.843293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:04.844305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:04.845090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 150/276] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:04.845798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.914428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23c15b-6ff7-4829-bd8e-3dfbb3e29593 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:04.943829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:04.960909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:04.962090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:04.962371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:04.962645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:04.963966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:04.965450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:04.967154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 127/277] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:04.967997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:04.994856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce232f-c2f6-4ef6-996f-614443a524fe None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.054462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.055285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.055603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.055895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.069420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:05.077730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:05.078694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 151/278] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:05.084071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.100898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138f23-398a-4001-942a-a7bdb372eab4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.154599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.155999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.156120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.158857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.160584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:05.161082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.163344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.176114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:05.177681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 128/279] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:05.186011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.252327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d1d86-e1d5-4d76-9f3a-0650e71f0ed7 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.273485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:05.294361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.295337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.295661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.295942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.296815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.298284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.320410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 152/280] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:05 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:05.321289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.356131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ab823-8884-41d0-b77b-dcf5f165f884 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.429844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.432305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.433063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.433728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.435331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:05.436105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.440078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.460231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 129/281] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:05.467379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.527233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456b1a0-77ec-4a1c-87cb-526d2d009d46 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.546195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:05.605852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.606768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.607330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.607330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] PATH_INFO: `/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.608913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] RBAC: Authorizing `identity:get_user(user_id=ad5b9dc286cf4d0fb42c1a91f9867369)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.610446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.615514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 153/282] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:05 2022] GET /identity/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:05.619385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.654387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd2643-55b7-40a5-936f-00adc8e50f01 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.676174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:05.706009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.706917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.707202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.707500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] PATH_INFO: `/v3/users/7a9e81d325174a799c6613936d8a23f1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.735133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] RBAC: Authorizing `identity:get_user(user_id=7a9e81d325174a799c6613936d8a23f1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.739165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.742505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 130/283] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:05 2022] GET /identity/v3/users/7a9e81d325174a799c6613936d8a23f1 => generated 293 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:05.750147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.804197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7727b-6b7f-43a5-9d95-619e3d49316c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:05.872621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:05.873537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:05.873865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:05.874148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:05.876003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:05.877529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:05.879713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 154/284] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:05 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:05.888872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:05.944709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364881bb-6334-40a8-a085-3e858ee8239e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:06.028709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:06.031073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:06.032256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:06.032744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:06.033190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:06.034833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, user.id=7a9e81d325174a799c6613936d8a23f1, scope.system=all) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:06.035417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:06.039250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677d833-828d-4958-b4cb-ee2cca324abd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:06.055772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 131/285] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Jul 21 12:01:05 2022] GET /identity/v3/role_assignments?user.id=7a9e81d325174a799c6613936d8a23f1&scope.system=all&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:06.176610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1207615-414e-49d0-bf5a-60dc6f5eaf55 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:06.266390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:06.268520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:06.269262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:06.270011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/ad5b9dc286cf4d0fb42c1a91f9867369/roles/8cf34c361caf421f8f85b4274ca9ac1a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:06.274185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, user_id=ad5b9dc286cf4d0fb42c1a91f9867369, role_id=8cf34c361caf421f8f85b4274ca9ac1a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:06.277527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20b70-44fc-4cdb-86cb-b1c0192a67f2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:06.313303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 155/286] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:05 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/users/ad5b9dc286cf4d0fb42c1a91f9867369/roles/8cf34c361caf421f8f85b4274ca9ac1a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:06.501159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0d0b4-067d-4453-8b71-8fbbef1d0075 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:06.501949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0d0b4-067d-4453-8b71-8fbbef1d0075 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:06.502122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0d0b4-067d-4453-8b71-8fbbef1d0075 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:06.505279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 132/287] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:06.515075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:06.516015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:06.516738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:06.551626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:06.554232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:06.723565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 156/288] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:06.764929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0d0b4-067d-4453-8b71-8fbbef1d0075 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:06.835930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:06.991620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0d0b4-067d-4453-8b71-8fbbef1d0075 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:07.074294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:07.075681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:07.076367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:07.076849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:07.104556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:07.119982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:07.121675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:07.125220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 133/289] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:07.132049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:07.242892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:07.417768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbca52a-9184-4477-897d-fc6dcb5e8fcc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:07.507674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:07.508269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:07.508553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:07.508824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:07.509607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:07.509910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:07.511530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:07.522016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 157/290] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:07.531813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:07.795424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:07.812884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc6b3-e77a-47ea-b1d7-02b428667625 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:07.893039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:07.894614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:07.895099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:07.895600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] PATH_INFO: `/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:07.897771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] RBAC: Authorizing `identity:get_user(user_id=70b0bc75a23c4e88ba4274ce5f77fb40)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:07.900547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:07.903791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 134/291] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:07 2022] GET /identity/v3/users/70b0bc75a23c4e88ba4274ce5f77fb40 => generated 297 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:07.912882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:08.044668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:08.051879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40271ba0-733f-49bd-b471-a195d3737d4d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:08.089770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:08.090447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:08.090731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:08.091008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:08.179452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:08.181066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:08.287377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:08.295663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 158/292] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:08.304914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:08.539119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a3f2c-1591-4e34-bdca-68805743619a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:08.599764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:08.600588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:08.600872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:08.601159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:08.608599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:08.610147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713282e-5eb0-4feb-b7ef-6d1c53d04709 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:08.612123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 135/293] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:08 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:08.626201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:08.749912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb5e5-1fc8-4550-8a0f-d31b4b1ef06a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:08.750396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb5e5-1fc8-4550-8a0f-d31b4b1ef06a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:08.750826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceb5e5-1fc8-4550-8a0f-d31b4b1ef06a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:08.754014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 136/294] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:08.764896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:08.765341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:08.765809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:08.819109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:08.821211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:08.888735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf6944-fb67-4a05-b317-8961b51b19ad None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:08.900059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:08.965758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:08.967396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:08.968191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:08.968964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:08.970967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=70b0bc75a23c4e88ba4274ce5f77fb40) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:08.971829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:08.983522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:08.983789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 137/295] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:09.001141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 159/296] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:08 2022] GET /identity/v3/role_assignments?user.id=70b0bc75a23c4e88ba4274ce5f77fb40&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 608 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:09.046107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:09.235218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22695a50-0fcc-41db-a594-5a840b44e227 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:09.299026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:09.300486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.300875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.301162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.320935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-24b94e93-d3b5-4217-9be5-bd309fd6fb8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:09.330885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:09.333518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:09.336759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 138/297] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:09.348978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:09.452714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2e47-703f-49cd-8057-33726903d0c6 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.453470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2e47-703f-49cd-8057-33726903d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.454140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2e47-703f-49cd-8057-33726903d0c6 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.458232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 139/298] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:09.464372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83c02-b71f-4ddb-a376-bdbfae040846 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.464831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83c02-b71f-4ddb-a376-bdbfae040846 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.465301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83c02-b71f-4ddb-a376-bdbfae040846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.486616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:09.502059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ecb83c02-b71f-4ddb-a376-bdbfae040846 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:09.504144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ecb83c02-b71f-4ddb-a376-bdbfae040846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:09.537287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:09.566115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96699228-a9ea-476b-a09f-96e28d4f85e2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:09.631450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:09.632591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.633071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.633546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.634933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:09.635571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:09.638106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:09.638881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 140/299] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:09.641664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81e5de-9223-4d2f-be50-bd83a0a1e00e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.642097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81e5de-9223-4d2f-be50-bd83a0a1e00e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.642494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81e5de-9223-4d2f-be50-bd83a0a1e00e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.644574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 141/300] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:09.655320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 160/301] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:09.658247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.658745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.659235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.661105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:09.684293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:09.685585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:09.779765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 142/302] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:09.779765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:09.808160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e55e-c619-4dcb-b3db-6e32b757b00a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:09.835128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:09.851956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:09.852896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:09.853252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:09.853527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] PATH_INFO: `/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:09.870622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] RBAC: Authorizing `identity:get_user(user_id=d32cf9c082fe42c78c4ec1e6bd18391d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:09.872311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:09.874398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 161/303] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:09 2022] GET /identity/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:09.875060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:09.925089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:09.958365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74a9a6-1e37-433a-9d45-7461fd8a7661 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.017670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.018951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.019418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.019835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.035929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ce7ee-b29f-47fb-be82-d8050760654a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.037957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:10.050720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:10.052617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:10.054267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:10.059621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 143/304] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:10.060981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.097239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.098264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.099374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.099374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.100801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.102657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.105223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 162/305] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:10.106020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.175934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:10.245636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6d4f3-fea0-442c-abef-938bd25f6c76 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.255404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc945e-511e-4525-90ec-120f1845894c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.296245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.296802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.297084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.297362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.298214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, user.id=d32cf9c082fe42c78c4ec1e6bd18391d, scope.system=all) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:10.298521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.299099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.299813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.300038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.300225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.300358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.301083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:10.301382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.302788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4cead-2a6c-4b9e-9026-98537ebda35b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.306267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 144/306] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Jul 21 12:01:10 2022] GET /identity/v3/role_assignments?user.id=d32cf9c082fe42c78c4ec1e6bd18391d&scope.system=all&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:10.307222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.312284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 163/307] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:10.313712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aa5f-c474-4f11-84c5-f9b2aeb564a8 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.313996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aa5f-c474-4f11-84c5-f9b2aeb564a8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.314254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713aa5f-c474-4f11-84c5-f9b2aeb564a8 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.315982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 164/308] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:10.320821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713aa5f-c474-4f11-84c5-f9b2aeb564a8 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.327948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:10.470839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48eb6-c6a0-48c2-b407-432c807a3049 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.478176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713aa5f-c474-4f11-84c5-f9b2aeb564a8 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.494643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:10.515693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.516416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.516700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.516983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] PATH_INFO: `/v3/groups` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.517718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.519237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.530014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.530885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.531160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.531442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] PATH_INFO: `/v3/users/7a9e81d325174a799c6613936d8a23f1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.533057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=7a9e81d325174a799c6613936d8a23f1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.534626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de938d6c-ea6d-4786-b690-e0bf3679b0c7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.543855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 165/309] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:10 2022] GET /identity/v3/users/7a9e81d325174a799c6613936d8a23f1 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:10.543855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.543855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.543855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.544463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 145/310] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Jul 21 12:01:10 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:10.549366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.573833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:10.576409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:10.684479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:10.735116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 166/311] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:10.754454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f87d-3edb-4ef8-9641-09b986abb3b5 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.760551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:10.817964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.818764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.819054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.819303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] PATH_INFO: `/v3/system/users/7a9e81d325174a799c6613936d8a23f1/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.821004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7a9e81d325174a799c6613936d8a23f1, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:10.823117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:10.837632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 146/312] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Jul 21 12:01:10 2022] PUT /identity/v3/system/users/7a9e81d325174a799c6613936d8a23f1/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:10.905621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35ea3c-d826-449e-a9ab-5e7456bd8946 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:10.920115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:10.952326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:10.953206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:10.953484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:10.953752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:10.967913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:10.975841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:10.977897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:10.979644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 167/313] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:10.983954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:11.103439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90e63-e58d-4a98-95d5-6a7ef519e711 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:11.136539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:11.147823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:11.147823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:11.147823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:11.147823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:11.148458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:11.148458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:11.150283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:11.175814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 147/314] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:11.178844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:11.383863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bd918-9e2e-4e48-9007-21f874c36b22 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:11.450657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:11.452265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:11.452752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:11.453233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] PATH_INFO: `/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:11.455672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] RBAC: Authorizing `identity:get_user(user_id=ad5b9dc286cf4d0fb42c1a91f9867369)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:11.458502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:11.462224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 168/315] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:11 2022] GET /identity/v3/users/ad5b9dc286cf4d0fb42c1a91f9867369 => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:11.470620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:11.640846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ca706-a455-42a5-b3f2-8d48954cebeb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:11.679074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:11.680190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:11.680672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:11.681256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:11.757080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:11.770102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:11.772170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:11.960188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 148/316] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:11.967951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:12.031147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:12.179704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1e5c9-ef9c-4d0e-ada8-e153006e0f20 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:12.242661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:12.244047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:12.244573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:12.245004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:12.247201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:12.249653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc3af0-e239-4110-817b-1ab4c56e5ce2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:12.253315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 169/317] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:11 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:12.259219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:12.390628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:12.406766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1f750-d287-42ea-b253-e4079bdfab92 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:12.473329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:12.474611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:12.475076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:12.475595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:12.477235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1, user.id=ad5b9dc286cf4d0fb42c1a91f9867369) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:12.477799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:12.480552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae475b6-2077-492f-9899-3cd4bdd527e2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:12.492800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 149/318] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:12 2022] GET /identity/v3/role_assignments?user.id=ad5b9dc286cf4d0fb42c1a91f9867369&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:13.716858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ca60-cde9-4d54-9828-c139d278f48e None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:13.717694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ca60-cde9-4d54-9828-c139d278f48e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:13.717694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ca60-cde9-4d54-9828-c139d278f48e None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:13.719557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 170/319] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:13.725413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96caace-c135-4d5a-8266-cb107f1aba7e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:13.726166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96caace-c135-4d5a-8266-cb107f1aba7e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:13.726305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96caace-c135-4d5a-8266-cb107f1aba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:13.745887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a96caace-c135-4d5a-8266-cb107f1aba7e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:13.747298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a96caace-c135-4d5a-8266-cb107f1aba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:13.825894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 150/320] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:13.849091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253ca60-cde9-4d54-9828-c139d278f48e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:13.963238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affe49a-1391-48e6-bdce-0150c98e663f None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:13.963726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affe49a-1391-48e6-bdce-0150c98e663f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:13.963899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253ca60-cde9-4d54-9828-c139d278f48e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:13.964524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affe49a-1391-48e6-bdce-0150c98e663f None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:13.966683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:13.971642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 151/321] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:13.976657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94a0c-ffac-4ec3-9b47-05fa6800aefa None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:13.977152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94a0c-ffac-4ec3-9b47-05fa6800aefa None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:13.977644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a94a0c-ffac-4ec3-9b47-05fa6800aefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.005523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-76a94a0c-ffac-4ec3-9b47-05fa6800aefa None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:14.007788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-76a94a0c-ffac-4ec3-9b47-05fa6800aefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:14.008494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.009322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.009601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.009890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.026769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:14.039994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:14.041794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:14.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:14.049401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 171/322] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:14.050565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.105145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 152/323] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:14.108477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc4994-e4d1-418d-ad55-2436fb252ae2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.108477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc4994-e4d1-418d-ad55-2436fb252ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.108477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc4994-e4d1-418d-ad55-2436fb252ae2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.108853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 153/324] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:14.114936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.115301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.115688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.133874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:14.135212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:14.149773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.205740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eb1ad-8c3e-46c1-8830-2e16cb7decb6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.210077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 154/325] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:14.211394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.251396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.252370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.252650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.252929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.253707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:14.254011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.255422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.264782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 172/326] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:14.265928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.295462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.372404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d168-dba0-4921-8f27-dbde685059d2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.374291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d98e1-ad41-4976-a0c6-71ef2b3e22e5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.412061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.412881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.413161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.413434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.414675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.416242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.417909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 173/327] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:14.418683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.433231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.434643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.435133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.435653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.454288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:14.469247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:14.470637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:14.472165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 155/328] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:14.473540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.488291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.533533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.545024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a8f81-ee4e-4610-8f1d-15e22d191723 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.590374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.591291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.591594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.591868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] PATH_INFO: `/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.593226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] RBAC: Authorizing `identity:get_user(user_id=d32cf9c082fe42c78c4ec1e6bd18391d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.594716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.596684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 174/329] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:14.597330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.611815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b404-923a-4ec3-ad89-5f5d0412f7d6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.652296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.653167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.653541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.653862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] PATH_INFO: `/v3/groups` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.654880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.656623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.673973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 156/330] 10.222.0.18 () {60 vars in 1134 bytes} [Thu Jul 21 12:01:14 2022] POST /identity/v3/groups => generated 226 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:14.675020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.757563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55be384-047a-4c57-8e52-38ec48a81058 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.814204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.815994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.816839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.817326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.817799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.819041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:14.819965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.822593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.838097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 175/331] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:14.844849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:14.884641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e44481-e7c3-499c-961e-935ba861ddef None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:14.889269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:14.937161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:14.938839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:14.939114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:14.939448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] PATH_INFO: `/v3/system/users/d32cf9c082fe42c78c4ec1e6bd18391d/roles/8cf34c361caf421f8f85b4274ca9ac1a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:14.941154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d32cf9c082fe42c78c4ec1e6bd18391d, role_id=8cf34c361caf421f8f85b4274ca9ac1a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:14.942646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:14.955651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 157/332] 10.222.0.18 () {58 vars in 1334 bytes} [Thu Jul 21 12:01:14 2022] PUT /identity/v3/system/users/d32cf9c082fe42c78c4ec1e6bd18391d/roles/8cf34c361caf421f8f85b4274ca9ac1a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:15.047909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2aa05-8813-475e-81ea-8457bd434d3a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:15.120339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:15.121719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:15.122136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:15.122639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] PATH_INFO: `/v3/users/7a9e81d325174a799c6613936d8a23f1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:15.125214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] RBAC: Authorizing `identity:get_user(user_id=7a9e81d325174a799c6613936d8a23f1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:15.128094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f35e84-75aa-46ca-98dc-7e2bbe21f66f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:15.130926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 176/333] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:14 2022] GET /identity/v3/users/7a9e81d325174a799c6613936d8a23f1 => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:15.141450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:15.316392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd46e6-76b6-4963-bf32-76cb09a493b3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:15.350469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:15.356380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:15.357885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:15.358545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:15.359315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:15.360265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] RBAC: Adding query filter params (role.id=f7a7a1e9e73d449ea34fa43e31b197f6, user.id=7a9e81d325174a799c6613936d8a23f1, scope.system=all) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:15.360933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:15.363428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a344dbb-d4f8-40bb-82e9-cfc9e54c9f12 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:15.379287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 158/334] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Jul 21 12:01:15 2022] GET /identity/v3/role_assignments?user.id=7a9e81d325174a799c6613936d8a23f1&scope.system=all&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:17.954404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee13e-8a40-4737-b184-bd113551f96b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:17.954957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee13e-8a40-4737-b184-bd113551f96b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:17.954957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee13e-8a40-4737-b184-bd113551f96b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:17.956498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 177/335] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:17.963034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:17.963649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:17.963649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:17.984883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:17.986171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:18.082297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 159/336] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:18.133119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ee13e-8a40-4737-b184-bd113551f96b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:18.323934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:18.334812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ee13e-8a40-4737-b184-bd113551f96b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:18.385595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:18.386425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:18.386711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:18.386986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:18.406473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-a60cee50-b08d-4f73-940c-cd84eb002ecd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:18.414306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:18.416167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:18.417745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 178/337] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:18.422073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:18.488819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f924-463f-4c51-ad26-a5c7b4262f1d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:18.489801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f924-463f-4c51-ad26-a5c7b4262f1d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:18.490609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f924-463f-4c51-ad26-a5c7b4262f1d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:18.493830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 179/338] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:18.503530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575480-97f4-4c8b-98ff-e8e7d3bf3d3d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:18.504442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575480-97f4-4c8b-98ff-e8e7d3bf3d3d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:18.505143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575480-97f4-4c8b-98ff-e8e7d3bf3d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:18.559546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-48575480-97f4-4c8b-98ff-e8e7d3bf3d3d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:18.559546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-48575480-97f4-4c8b-98ff-e8e7d3bf3d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:18.570742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46f850-90d3-4645-b25b-25d35fbc694f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:18.610307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:18.659665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:18.660354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:18.660642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:18.660900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:18.661674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:18.661976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:18.663389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:18.687117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 160/339] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:18.693711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:18.729661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 180/340] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:18.781752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c398b-16b9-458a-ae4c-258370eb0a81 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:18.782665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c398b-16b9-458a-ae4c-258370eb0a81 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:18.783454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c398b-16b9-458a-ae4c-258370eb0a81 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:18.786748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 181/341] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:18.795437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c398b-16b9-458a-ae4c-258370eb0a81 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:18.904087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cac22-61b2-4369-8a9f-949dfba6f24e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:18.953555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.007616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.010350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.011217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.011936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] PATH_INFO: `/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.012260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c398b-16b9-458a-ae4c-258370eb0a81 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.016815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] RBAC: Authorizing `identity:get_user(user_id=d32cf9c082fe42c78c4ec1e6bd18391d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:19.019732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e27ac-d5bf-4c73-ad2d-570e1ca3559f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:19.023413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 161/342] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:01:18 2022] GET /identity/v3/users/d32cf9c082fe42c78c4ec1e6bd18391d => generated 293 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:19.025859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c904df-3ac2-4218-8a8f-d07918af6b16 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.026565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c904df-3ac2-4218-8a8f-d07918af6b16 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.027195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c904df-3ac2-4218-8a8f-d07918af6b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.065156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-24c904df-3ac2-4218-8a8f-d07918af6b16 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:19.067011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-24c904df-3ac2-4218-8a8f-d07918af6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:19.072199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.073158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.073434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.073706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.088246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-e0624745-36c9-4a09-947c-85b0ae6c8af9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:19.096194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:19.097226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:19.098076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:19.100518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 182/343] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:19.102191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e49cc-f942-4565-8c8d-d0d226cf1560 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.102604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e49cc-f942-4565-8c8d-d0d226cf1560 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.103014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e49cc-f942-4565-8c8d-d0d226cf1560 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.108799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 183/344] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:19.110027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e49cc-f942-4565-8c8d-d0d226cf1560 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:19.174497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.220616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 162/345] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:19.223246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.224065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.224825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.259426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:19.262021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:19.364686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e49cc-f942-4565-8c8d-d0d226cf1560 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.404329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 163/346] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:19.406683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:19.438534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.440205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.441337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.442054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.443953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] RBAC: Adding query filter params (role.id=8cf34c361caf421f8f85b4274ca9ac1a, user.id=d32cf9c082fe42c78c4ec1e6bd18391d, scope.system=all) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:19.444882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:19.448315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:19.473402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.477931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 184/347] 10.222.0.18 () {56 vars in 1308 bytes} [Thu Jul 21 12:01:19 2022] GET /identity/v3/role_assignments?user.id=d32cf9c082fe42c78c4ec1e6bd18391d&scope.system=all&role.id=8cf34c361caf421f8f85b4274ca9ac1a => generated 510 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:19.479964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:19.586290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf653b9-bc23-41a3-ae84-bf7a136954f4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.598306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.631818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.632790 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.633223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.633667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.634603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:19.635128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:19.636745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a3b1e-de19-4973-930f-9c4cbe0b0412 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:19.646703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 164/348] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:19.648465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b519b-3fa7-44ec-8ebd-b961bc010f8c None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.649063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b519b-3fa7-44ec-8ebd-b961bc010f8c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.649550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b519b-3fa7-44ec-8ebd-b961bc010f8c None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.651206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 165/349] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:19.656272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b519b-3fa7-44ec-8ebd-b961bc010f8c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:19.663591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4a8a9-a3c3-4597-b08e-95996f80c95d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.730837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.732517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.733202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.733882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.754091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:19.770927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:19.772706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 185/350] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:19.774267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:19.793517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b519b-3fa7-44ec-8ebd-b961bc010f8c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.845922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.847219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.848254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.848254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.868707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-78f91ba0-f88f-4094-944c-1f579228b699 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:19.879117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:19.880258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:19.881051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:19.881642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.881714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 166/351] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:19.881777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.882059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.882330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:19.916939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:19.920435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:19.941956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87a97a-560c-4dea-972e-3f9a66adf48c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:19.964461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:19.990614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:19.991174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:19.991524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:19.991811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.027680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 167/352] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:20.029212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.129015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.131975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.164410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3e0b6-8055-4541-a089-ce51b89c2c52 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.221564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.221564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.221564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.221564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.222202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:20.222202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.223854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.228592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:20.237957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 168/353] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:20.239313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.280007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 186/354] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:20.280482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.288060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:20.424285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80cd0a-c797-4ec0-be82-a0b6dcb06d7b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.426078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f017a81-9826-4236-bd9a-e3a128f2a89d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.482432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.483270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.483590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.483896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.486657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.487636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.489495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.489495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.489798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:20.490170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.494771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.496638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:20.504889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:20.506833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 187/355] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:20.508745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.516242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 169/356] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:20.524537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.533943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:20.652185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470404ad-e1f0-4c9c-b742-0ab439c5c3e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.656932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:20.679174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0076-030a-47af-a92b-b1ec43da95c3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.714698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.715379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.715758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.716319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.737286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.737660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.737877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.738159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.740320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.742586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.745949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 170/357] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:20.746963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.816550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.818376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.847449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:20.918826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560de0ac-40b4-4e97-9d87-b838b949d9b1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:20.971539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:20.971539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.971539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.972019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.973012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:20.973483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:20.975313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187774c-4a09-4f14-8d8d-f917378bf762 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:20.981526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 188/358] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:20.983463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:20.991527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 171/359] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:20.992961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f9-8d1e-4442-8481-8ef6004e2636 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:20.993258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f9-8d1e-4442-8481-8ef6004e2636 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:20.993542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4729f9-8d1e-4442-8481-8ef6004e2636 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:20.995301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 172/360] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:20.995886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729f9-8d1e-4442-8481-8ef6004e2636 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:21.013754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:21.138206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4729f9-8d1e-4442-8481-8ef6004e2636 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:21.145059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cffd78-2e6d-493b-9038-be16194f3537 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:21.197276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:21.198224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.198560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.198928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.199358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:21.211689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:21.214645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:21.216902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 173/361] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:21.217985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:21.232034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:21.232986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.233457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.233912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.364246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:21.375550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:21.375550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19987b74-df06-44bb-aa45-39f33d0d4f49 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:21.408799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d2e97-6f16-4931-ae05-47242706603a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:21.471424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:21.473247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.473965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.474334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.477036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:21.479240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:21.482038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 174/362] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:21 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:21.483794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:21.547130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 189/363] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:21.548655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.549051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.549454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.582931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:21.586825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:21.616018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:21.666978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177c043-5153-4fce-b52c-3c22660325a0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:21.709673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 190/364] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:21.711440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:21.735807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:21.737188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.737628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.738048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.770145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:21.861012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:21.863355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:21.863684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaff2c0-a6f9-4141-ac85-959409ef76e1 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:21.912033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:21.912717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:21.912997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:21.913270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:21.914135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=43f8b4157afc45d28ef515833ee374d1) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:21.914521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:21.916704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:21.923570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 191/365] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:21 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:21.924328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:21.953335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.035193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 175/366] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:22.037241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.101650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f57ea-d298-4195-86d5-5ee982fc1832 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.117671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.170466 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.172017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.172525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.173181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.175648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.178054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.186008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 192/367] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:21 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.186008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.196209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ef47c-d513-40e9-8f15-47fb7ab042d0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.235211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.236135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.236414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.236683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.238234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.239766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.247620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 176/368] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.247620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.330645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.363422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4e5ea-f628-4642-aff8-1043a9c17028 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.409242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd9c68-0f0e-4695-8d6e-61ba10fe1066 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.419576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.420541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.420820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.421095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.431776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.435142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:22.443233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:22.444778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 193/369] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:22.446407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.478890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.480300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.480760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.481213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.483279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.485755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.488701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 177/370] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.490065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.587024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35ce0-3d80-4d46-8fe1-bcedb73a27da None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.627450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.628253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.628539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.628824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.629659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=43f8b4157afc45d28ef515833ee374d1) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:22.629971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.631735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.634798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8aeba-8a1c-4948-a5aa-f6ce23a75706 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.638944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 194/371] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.639825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.667194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.688848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.691048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.691587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.692009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.693643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.695336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 178/372] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.700417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.778710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:22.816080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2de7b1-da47-47cb-8105-4e85cc8fd273 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.859468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27535637-7548-43ec-bc40-3b999c54d47b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:22.864003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.865174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.865504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.865935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.867320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:22.869226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.871836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.887370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 195/373] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.888726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.927462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:22.929886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:22.929886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:22.930201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:22.932458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:22.935168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:22.938979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 179/374] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:22.940394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:22.993812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:23.044187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b527dd2-82b3-45cd-ab73-c828c9a32a73 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:23.086901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:23.088177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:23.088768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:23.089222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:23.090313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:23.090836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:23.092617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:23.100477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 196/375] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:23.101780 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:23.121491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f64e68-6362-4f23-a0ab-c22b1a1ef02b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:23.168183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:23.208749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:23.209401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:23.209619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:23.209914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:23.233081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68ead9-eedf-4196-9c9a-c8cdfa5055cb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:23.276905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:23.282926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:23.286585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:23.288070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:23.288734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:23.291232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:23.292714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:23.295180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:23.312903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 197/376] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:23 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:23.352228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:23.354285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:23.507833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 180/377] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:23.516706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:23.677677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f5134-ce6a-44df-8469-1b075bc8e61b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:23.746407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:23.747321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:23.747642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:23.747923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] PATH_INFO: `/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:23.752480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] RBAC: Authorizing `identity:get_project(project_id=8a4579ac3a4640e986aabfbb2b1ede46)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:23.754001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:23.755918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 198/378] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:23 2022] GET /identity/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:23.761953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:23.792646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:23.891093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee2ce5-be26-41a4-a1ea-59fb3c77872a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:23.935207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:23.936505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:23.937017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:23.937508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] PATH_INFO: `/v3/groups/bd62702b95174cabaca596d85eaf0d9f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:23.943799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] RBAC: Authorizing `identity:get_group(group_id=bd62702b95174cabaca596d85eaf0d9f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:23.945619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526b060-637d-43a5-9e24-4b9ccf05a9cd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:23.947722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 181/379] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:23 2022] GET /identity/v3/groups/bd62702b95174cabaca596d85eaf0d9f => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:23.956669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:24.059812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:24.111766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c621fa-2281-44c3-b01c-20943d843b63 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:24.167372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:24.168108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:24.168398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:24.168683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:24.169557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] RBAC: Adding query filter params (group.id=bd62702b95174cabaca596d85eaf0d9f, role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=8a4579ac3a4640e986aabfbb2b1ede46) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:24.169879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:24.171403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c57fd-6dac-4bfe-acda-4d99c93a7982 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:24.178781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 199/380] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:23 2022] GET /identity/v3/role_assignments?group.id=bd62702b95174cabaca596d85eaf0d9f&scope.project.id=8a4579ac3a4640e986aabfbb2b1ede46&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:24.265530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b34e1-72ff-4528-8939-0253bcdc02a0 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:24.266547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b34e1-72ff-4528-8939-0253bcdc02a0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:24.267021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b34e1-72ff-4528-8939-0253bcdc02a0 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:24.269217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 182/381] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:24.276006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:24.276533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:24.276533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:24.300889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:24.302256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:24.401500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 200/382] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:24.424311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b34e1-72ff-4528-8939-0253bcdc02a0 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:24.549211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:24.577398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b34e1-72ff-4528-8939-0253bcdc02a0 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:24.631141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:24.632409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:24.632908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:24.633388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:24.646118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:24.654686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:24.655644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:24.656415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:24.663611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 183/383] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:24.667416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:24.775872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:24.832424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a189-b071-4ab5-9933-728b540e76d9 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:24.884166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:24.884974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:24.885315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:24.885635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:24.886440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:24.886755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:24.888190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:24.906376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 201/384] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:24.924726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:25.076808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ea0a-3af5-4096-b248-d2b89ef88e59 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:25.119156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:25.120476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.120970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.121457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.122937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:25.124687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:25.126593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 184/385] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:24 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:25.133389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:25.365609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955c9e4-184d-420e-ae68-5cc3752309ba None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:25.387898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:25.411328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:25.412311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.412596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.412866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.414181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:25.415697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27091992-e7fc-4a37-8c74-c8fd5ec1ff55 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:25.418443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 202/386] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:25 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:25.424381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:25.446470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:25.576555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd324fba-dfa7-4abd-9ebe-1204ea6fc684 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.576942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd324fba-dfa7-4abd-9ebe-1204ea6fc684 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.577149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd324fba-dfa7-4abd-9ebe-1204ea6fc684 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.578578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 203/387] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:25.579695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ba396-56c7-4663-8354-8787b3fd7358 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:25.588098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.588371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.588823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.619782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:25.620922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.621348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.621780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.624527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:25.624670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:25.625766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:25.626487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:25.654275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 185/388] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Jul 21 12:01:25 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:25.717891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 204/389] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:25.757931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:25.869390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:25.914051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd6779-f73c-49fd-9316-a933ffec2bfe None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:25.959687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:25.961372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:25.961890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:25.962357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:25.977523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-37a1eda4-5b48-46da-814e-d187bb61ee56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:25.986688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:25.988007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:25.988813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:25.992898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 186/390] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:26.003530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.179010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79e10-cf9d-44cf-b5ce-5d3b5a97f344 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:26.213108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:26.221539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af820e64-8bcf-46bd-95a4-0935f21680c6 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.222216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af820e64-8bcf-46bd-95a4-0935f21680c6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.223595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af820e64-8bcf-46bd-95a4-0935f21680c6 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.225305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 187/391] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:26.232329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2a35-b10f-4c1e-bda0-9c5f74765b03 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.234339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2a35-b10f-4c1e-bda0-9c5f74765b03 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.235642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f2a35-b10f-4c1e-bda0-9c5f74765b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.250103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:26.251201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.256304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.256882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.257950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:26.258484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:26.260217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:26.271798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fd1f2a35-b10f-4c1e-bda0-9c5f74765b03 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:26.274423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fd1f2a35-b10f-4c1e-bda0-9c5f74765b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:26.279385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 205/392] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:26.290185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.445337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 188/393] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:26.479841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183027-5501-4735-9dd1-d9603e757102 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.481564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183027-5501-4735-9dd1-d9603e757102 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.481846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183027-5501-4735-9dd1-d9603e757102 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.483550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 189/394] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:26.484108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12183027-5501-4735-9dd1-d9603e757102 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.497092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ee32b-d3d5-43a6-bcb0-d9bda2581c2c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:26.533283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:26.534164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.534442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.534709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.536105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:26.537588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617350ad-378d-429f-b585-ce86421a3ee2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:26.543339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 206/395] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:26 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:26.543339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.543339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.543339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.559794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:26.561053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:26.609610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:26.612848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:26.643629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 207/396] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:26.645076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.695810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12183027-5501-4735-9dd1-d9603e757102 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:26.762095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:26.763957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.764842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.765718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.776158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fb2e4-f016-4cb6-b318-e1c264f8f17a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:26.788773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:26.805184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:26.806843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:26.808286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:26.813316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 190/397] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:26.815229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.822065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:26.823045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.823362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.823673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.826306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:26.826679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:26.828594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 208/398] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:26 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:26.829450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:26.885870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:26.949010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10730387-25de-4db2-ab53-ed47392bccb8 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:26.988220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:26.988907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:26.989182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:26.989452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:26.990216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:26.990519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:26.992080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.006326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 209/399] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:27.007294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.029935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1cd2-0627-4ec6-8ce1-16cd1961ebbc None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.070642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:27.098556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.100412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.101055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.101661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.122732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:27.131436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d300f3-9d2f-4928-866e-fa80524b2743 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:27.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:27.142105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:27.142878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:27.148532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 191/400] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:27.148532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.150035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:27.180008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.181006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.181282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.181631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.184479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] RBAC: Authorizing `identity:create_grant(project_id=43f8b4157afc45d28ef515833ee374d1, group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:27.185998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.203853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 210/401] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Jul 21 12:01:27 2022] PUT /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:27.203853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.338171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15cdd-7b0a-4877-bfd7-de8ed00bba42 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.364324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e10ce-1e19-4217-8ba7-c87bd1e3c24a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.381852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.382537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.382818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.383090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.383923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:27.384281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:27.385809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb93882-3524-4f53-959f-e526ba70de8b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.395603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 211/402] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:27.397132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1fc64-0471-4a96-aa49-ccb36e0371d7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.397320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1fc64-0471-4a96-aa49-ccb36e0371d7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.397566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1fc64-0471-4a96-aa49-ccb36e0371d7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.399086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 212/403] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:27.405439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1fc64-0471-4a96-aa49-ccb36e0371d7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.434128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.435655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.436090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.436531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.438847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:27.441341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff4428-c146-490b-9e91-139710004f24 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.444722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 192/404] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:27.445362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:27.447472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.448047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.448363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.473663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:27.474846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:27.552561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:27.575028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1fc64-0471-4a96-aa49-ccb36e0371d7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.597914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 193/405] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:27.599415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.645471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.647359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.648143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.648845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.651471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:27.654404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.657893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 213/406] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:27.663732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.820907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99094fa-a51a-413d-92fa-b5becf521961 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.846092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:27.853769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2299-abf4-460f-a617-a5e06266db21 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:27.898364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.900108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.900928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.901586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.904591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:27.907443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:27.917938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 194/407] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:27.917938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:27.925681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:27.927423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:27.928168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:27.928857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:27.948024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:27.963646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:27.965385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:27.966899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:27.971697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 214/408] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:27.973303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.002013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:28.093863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f85c8-ecb5-43f8-801c-33133c1e2e20 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.140736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.141874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.142717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.142868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.144747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:28.146422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:28.148877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 195/409] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:27 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:28.151531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.163111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd73561-3c89-4899-a8d9-ef0f96d4e9f4 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.218036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.219451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.219883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.220273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.222886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:28.225072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7cd49-3531-4880-a266-68e2849c4f6a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:28.242459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 215/410] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Jul 21 12:01:27 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:28.243737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b4327-7a3b-4c7d-b3e7-5dc6dfcf9d38 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.244026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b4327-7a3b-4c7d-b3e7-5dc6dfcf9d38 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.244303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b4327-7a3b-4c7d-b3e7-5dc6dfcf9d38 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.250283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 216/411] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:28.251319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b4327-7a3b-4c7d-b3e7-5dc6dfcf9d38 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.286324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:28.317271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979ec1-7674-4a71-95e8-144ee5241a27 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.337622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:28.387695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.390178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.390961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.391688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.394733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:28.395954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:28.400145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22611a07-ba8c-485a-996f-30fb0813d7c2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:28.418877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 196/412] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:28.420509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.420855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.421139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.433292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b4327-7a3b-4c7d-b3e7-5dc6dfcf9d38 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.442287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:28.443649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:28.515744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.517744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.518451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.519223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.523283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1, group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:28.526236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:28.551551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 197/413] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:28.553193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.560974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 217/414] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Jul 21 12:01:28 2022] PUT /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:28.593789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.849390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:28.876252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343927d9-6a7b-4ad5-8df1-e2f121941d68 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.890471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:28.899976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44b2d5-05d6-4e39-b0be-53c1c4904596 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:28.938476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.939905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.940341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.940797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] PATH_INFO: `/v3/groups/bd62702b95174cabaca596d85eaf0d9f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:28.946988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] RBAC: Authorizing `identity:get_group(group_id=bd62702b95174cabaca596d85eaf0d9f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:28.949124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:28.951527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 198/415] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:28 2022] GET /identity/v3/groups/bd62702b95174cabaca596d85eaf0d9f => generated 229 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:28.959380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:28.992016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:28.993399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:28.993883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:28.994422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.031604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:29.054033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:29.055794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:29.057374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:29.058404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 218/416] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:29.062783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:29.225441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec4d8-6120-41c5-9f82-82efa3bac47f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:29.400889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:29.402409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:29.402894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:29.403355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] PATH_INFO: `/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.408632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] RBAC: Authorizing `identity:get_project(project_id=8a4579ac3a4640e986aabfbb2b1ede46)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:29.411309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:29.425061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 199/417] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:28 2022] GET /identity/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46 => generated 344 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:29.444630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:29.547936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:29.550753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800508c8-2dd7-4556-bd4e-748be20bf1bc None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:29.645143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023af4-6a76-443e-8120-1caf76e6af5b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:29.688209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:29.690056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:29.690056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:29.690056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.693215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] RBAC: Adding query filter params (name=member) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:29.693742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:29.699224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4f5c-961f-4c34-b9cd-618c469f2984 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:29.731873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:29.735888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 219/418] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 674 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:29.738377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b643-6a27-4463-a676-600728f90500 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:29.738842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b643-6a27-4463-a676-600728f90500 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:29.739287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b643-6a27-4463-a676-600728f90500 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.750035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 220/419] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:29.751612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b643-6a27-4463-a676-600728f90500 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:29.779794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:29.781030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:29.781461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:29.781900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] PATH_INFO: `/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46/groups/bd62702b95174cabaca596d85eaf0d9f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.787277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] RBAC: Authorizing `identity:create_grant(project_id=8a4579ac3a4640e986aabfbb2b1ede46, group_id=bd62702b95174cabaca596d85eaf0d9f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:29.789095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76c52b-544c-4e90-9535-47d0952c41af None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:29.831619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 200/420] 10.222.0.18 () {58 vars in 1442 bytes} [Thu Jul 21 12:01:29 2022] PUT /identity/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46/groups/bd62702b95174cabaca596d85eaf0d9f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:29.834021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:29.834482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:29.834964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:29.900759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:29.902660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:30.069030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b643-6a27-4463-a676-600728f90500 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:30.175362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:30.176096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.176380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.176660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.215647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 201/421] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:30.257054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:30.258502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:30.272899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:30.274425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:30.346363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:30.474549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 221/422] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:30.487435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:30.525109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e426ab-0b72-48cd-be93-e39c78a19c1e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:30.591527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:30.591527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.591527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.592018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.618334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:30.630822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:30.632491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:30.634040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 202/423] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:30.645255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:30.709686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:30.713876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591a11a-1c6b-4241-9dfd-4b09fe5780dc None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:30.760850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:30.762017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.762373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.762731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.764232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:30.766074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9c4aa-5870-4a79-b46d-94278d3aaca7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:30.768480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 222/424] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:30 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:30.770189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9404a6e-a185-4125-a358-4776b1062987 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.770465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9404a6e-a185-4125-a358-4776b1062987 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.770744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9404a6e-a185-4125-a358-4776b1062987 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.772444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 223/425] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:30.776477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9404a6e-a185-4125-a358-4776b1062987 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:30.783673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:30.798713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694152-430e-4e4e-8af1-91777cbc2c3a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:30.845025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:30.845724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.846003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.846275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.847747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:30.848054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:30.849489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07ed4-902b-405c-bd8d-a4f968e37ca6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:30.869686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 203/426] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:30.872332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.873340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:30.873995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:30.898027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:30.899755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:30.953564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9404a6e-a185-4125-a358-4776b1062987 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:30.997998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:30.999421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:30.999953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.000424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.002085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:31.003936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:31.006278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 224/427] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:30 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:31.007405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.043698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 204/428] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:31.045326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.054786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:31.134853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29a9d7-af82-4ce2-b5a9-3a759cebcbd9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:31.187189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:31.189166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.189969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.190519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.234429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:31.264474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc126a-8a26-49b6-92ff-193a322894e4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:31.295326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:31.298598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:31.331241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:31.332418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.333024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.333609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.335353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=43f8b4157afc45d28ef515833ee374d1) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:31.336406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:31.339581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:31.348833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:31.351597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 205/429] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 611 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:31.353230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.431976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 225/430] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:31.437908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.534035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2522be-1608-4e2a-90ab-c73fbd3ad404 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:31.595616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:31.597130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.597554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.598090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.617232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:31.626893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a137a8-85a5-4e02-bd40-0ba13ed1f2b5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2b33b805-943e-4491-a844-9badce1b7b95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:31.630888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:31.631921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 12:01:31.632691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:31.635062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 206/431] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:31.636893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a4965-7f67-436e-b8fa-1cc0dec0cdb5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.637468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a4965-7f67-436e-b8fa-1cc0dec0cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.638027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a4965-7f67-436e-b8fa-1cc0dec0cdb5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.641031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 207/432] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:31.643355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a4965-7f67-436e-b8fa-1cc0dec0cdb5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.663338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:31.683154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:31.684491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.684992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.685457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] PATH_INFO: `/v3/projects/43f8b4157afc45d28ef515833ee374d1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.688037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] RBAC: Authorizing `identity:get_project(project_id=43f8b4157afc45d28ef515833ee374d1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:31.690170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef5f-8723-4ffb-ad5c-bd404550b920 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:31.693795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 226/433] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/projects/43f8b4157afc45d28ef515833ee374d1 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:31.696020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.696574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.697162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.706780 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:31.724698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:31.728129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:31.832629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a4965-7f67-436e-b8fa-1cc0dec0cdb5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:31.854805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 227/434] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:31.856344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:31.900391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:31.900391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:31.900391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:31.900391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:31.900993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:31.900993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:31.905792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:31.920007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 208/435] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:31.921023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.009392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.038922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace25f3-b914-4aa2-90b5-219907fccd60 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.046761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bee5a7-e437-4015-9103-a2a633f84f65 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.097140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.098377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.098852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.099299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.099851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.101057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.101314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.101818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.102312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.103521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:32.105569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 228/436] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:32.106661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.110135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.126864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:32.142898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:32.143996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:32.144786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:32.145370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 209/437] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:32.146756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.257077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a2b77-18f6-4969-ac3a-e80fb68332bb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.292279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38549-1afd-4568-bf58-133a6a0bb826 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.299095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.299841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.300164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.300488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.330467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.339661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.340777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.341434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.341976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.343174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=43f8b4157afc45d28ef515833ee374d1) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:32.343750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.345702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:32.353479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 210/438] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=43f8b4157afc45d28ef515833ee374d1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:32.354925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.409021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.411670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:32.415168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.515882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13379b2-33c0-4973-9245-943fe89e2df5 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.571404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.572454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.572954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.573465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.574862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:32.575362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.577913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:32.581026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 229/439] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:32.588966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.595925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 211/440] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:32.605252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.704910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.750425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828606dc-01ff-437a-a03f-9a6a4438c740 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.753382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:32.776620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd6345-7bbe-4497-813a-6450c3b7e6be None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:32.818233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.820054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.821159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.821642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.824590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.825662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:32.830519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 230/441] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:32.836574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:32.839930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:32.840665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:32.840945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:32.841217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:32.942605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:32.944191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aef1fe-edb3-4696-afa9-4acb4df147c8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.003915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e65fe-eaef-4e44-af3d-8e990341ed31 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.022615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.042556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.043442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.043757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.044029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.045293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.046770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0592d-ee29-45b1-b851-dbf93967dcd6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.048482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 231/442] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:33.052957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39128a-966f-4cd2-9de7-5bd47612b2d3 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.053313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39128a-966f-4cd2-9de7-5bd47612b2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.053493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39128a-966f-4cd2-9de7-5bd47612b2d3 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.054871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 232/443] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:33.055686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39128a-966f-4cd2-9de7-5bd47612b2d3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.076185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 212/444] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:33.077561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459a05-788f-4987-8182-e9177b197500 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.077851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459a05-788f-4987-8182-e9177b197500 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.078130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459a05-788f-4987-8182-e9177b197500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.096062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b1459a05-788f-4987-8182-e9177b197500 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:33.097293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b1459a05-788f-4987-8182-e9177b197500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:33.111116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.175784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 213/445] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:33.176667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459a05-788f-4987-8182-e9177b197500 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.179557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39128a-966f-4cd2-9de7-5bd47612b2d3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.224200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.225356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.225759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.226430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.227591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=f7a7a1e9e73d449ea34fa43e31b197f6, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:33.227986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.229828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.236781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 233/446] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=f7a7a1e9e73d449ea34fa43e31b197f6 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:33.237628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.287303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.316014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459a05-788f-4987-8182-e9177b197500 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.377637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.378955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.379405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.379898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] PATH_INFO: `/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.382113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] RBAC: Authorizing `identity:get_project(project_id=7e17dfe4dab34fc3875edcd4f5fde1a1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.384711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.387861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 214/447] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/projects/7e17dfe4dab34fc3875edcd4f5fde1a1 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:33.396135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.411354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c5cd3-6380-4344-8b28-b71b96f31bc3 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.413721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.459844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.461027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.461315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.461605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.474518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:33.483859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:33.485491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:33.486743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 234/448] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:33.492223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.554462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf88a-f784-409e-9beb-7bdc8c8fda56 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.598251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.599770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.600206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.600672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] PATH_INFO: `/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.604111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] RBAC: Authorizing `identity:get_group(group_id=1fe9bef7c01b433c8ae7b9cb2b5a481d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.606581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.609533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 215/449] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/groups/1fe9bef7c01b433c8ae7b9cb2b5a481d => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:33.616111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.650821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.657971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9100b2-a435-4f26-b4f9-10a51add16b5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.697591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.698399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.698677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.698951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.699772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:33.700081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.701487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.710514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 235/450] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:33.717475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:33.742271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0212bb-9592-4f37-82d0-6e347517ff56 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.745543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:33.780774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.781445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.781729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.782008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.782858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] RBAC: Adding query filter params (group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d, role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:33.783169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.784692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.791782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 216/451] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/role_assignments?group.id=1fe9bef7c01b433c8ae7b9cb2b5a481d&scope.project.id=7e17dfe4dab34fc3875edcd4f5fde1a1&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:33.833632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8741a-dc9d-442e-a797-548439d40cb5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:33.874105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:33.874781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:33.875056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:33.875330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:33.956014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:33.957417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:33.985420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:34.071907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 236/452] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 12:01:34.077820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:34.181563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:34.197356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99db4-0b85-41a1-b93e-6fcaf6ac3cd4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:34.234247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:34.235054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:34.235336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:34.235649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] PATH_INFO: `/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:34.236928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] RBAC: Authorizing `identity:get_project(project_id=8a4579ac3a4640e986aabfbb2b1ede46)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:34.238501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:34.240286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 217/453] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:01:34 2022] GET /identity/v3/projects/8a4579ac3a4640e986aabfbb2b1ede46 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:34.246349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:34.355524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd189f4-f672-48bd-a26b-6e4dedf51f3d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:34.385017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:34.398684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:34.399690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:34.400018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:34.400400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] PATH_INFO: `/v3/groups/bd62702b95174cabaca596d85eaf0d9f` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:34.401802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] RBAC: Authorizing `identity:get_group(group_id=bd62702b95174cabaca596d85eaf0d9f)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:34.403369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cff96f-4e89-4286-b1bf-3ea3fe4be892 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:34.405290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 237/454] 10.222.0.18 () {56 vars in 1180 bytes} [Thu Jul 21 12:01:34 2022] GET /identity/v3/groups/bd62702b95174cabaca596d85eaf0d9f => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:34.412025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:34.532365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802d96-0411-4ea0-953b-dee03c5520ed None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:34.569971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:34.570633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:34.570913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:34.571187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:34.572106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] RBAC: Adding query filter params (group.id=bd62702b95174cabaca596d85eaf0d9f, role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=8a4579ac3a4640e986aabfbb2b1ede46) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:34.572416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:34.573890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342a4d-0dad-45fa-b84f-9abea16b7678 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:34.580686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 218/455] 10.222.0.18 () {56 vars in 1376 bytes} [Thu Jul 21 12:01:34 2022] GET /identity/v3/role_assignments?group.id=bd62702b95174cabaca596d85eaf0d9f&scope.project.id=8a4579ac3a4640e986aabfbb2b1ede46&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:37.351175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2cf6e-e778-4a71-9f84-d441195ff27f None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:37.352249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2cf6e-e778-4a71-9f84-d441195ff27f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:37.352723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2cf6e-e778-4a71-9f84-d441195ff27f None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:37.354604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 238/456] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:37.361892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d6aaf-71ad-4129-9e24-60202aed8a25 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:37.363302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d6aaf-71ad-4129-9e24-60202aed8a25 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:37.364044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d6aaf-71ad-4129-9e24-60202aed8a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:37.401379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-899d6aaf-71ad-4129-9e24-60202aed8a25 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:37.403813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-899d6aaf-71ad-4129-9e24-60202aed8a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:37.433780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:37.555793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 219/457] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:37.593710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2cf6e-e778-4a71-9f84-d441195ff27f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:37.852594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2cf6e-e778-4a71-9f84-d441195ff27f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:37.931602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2d94-11cc-45ac-a366-3c50adebc177 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:37.932250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2d94-11cc-45ac-a366-3c50adebc177 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:37.932714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2d94-11cc-45ac-a366-3c50adebc177 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:37.934708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 220/458] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:37.940520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:37.941275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:37.941753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:37.955807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:37.957126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:37.957593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:37.958090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:37.963386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:37.965008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:37.991002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:37.993848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.047875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 221/459] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:38.061147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:38.062056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:38.062872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:38.062872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:38.062872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:38.062872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:38.064622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 239/460] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:38.069467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.078437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.164134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:38.187904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c2d5f-121d-4e55-ba41-de717eb6e35c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:38.228490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:38.296129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:38.297251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.297723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.298213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.299622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:38.300163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:38.302539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.344299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 222/461] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:38.352280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.367333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b458ad-befc-4864-9217-0e0fdac65675 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:38.422336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:38.424216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.424494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.424761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.437642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:38.439526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-7af4b81b-6b3b-4322-bf70-c67c20b56e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:38.460298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:38.461576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:38.462390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:38.462390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:38.462390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:38.462390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:38.462390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 240/462] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:38.463630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cbdf-fa7e-4302-8df6-def34d26d9e2 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.464121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cbdf-fa7e-4302-8df6-def34d26d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.464519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cbdf-fa7e-4302-8df6-def34d26d9e2 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.466325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 241/463] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:38.469365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87cbdf-fa7e-4302-8df6-def34d26d9e2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.528032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf906b3-0321-4392-a0e1-f69a4fbd2c40 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:38.578515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:38.581471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:38.582612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.583119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.583665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.585049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:38.586501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87cbdf-fa7e-4302-8df6-def34d26d9e2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:38.588243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b824c69-de71-40e6-89bb-f7c638f8142d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.602158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:38.628558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 223/464] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:38 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:38.628558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749f59-175c-4462-9ee0-1c34685e119a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.628558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749f59-175c-4462-9ee0-1c34685e119a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.628558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749f59-175c-4462-9ee0-1c34685e119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.631704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:38.632406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.632867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.633095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.633873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:38.634175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:38.636005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385544a-c1f3-4518-8728-6ecabbb09a95 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.651703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 242/465] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:38.656757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-db749f59-175c-4462-9ee0-1c34685e119a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:38.657977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad33845-69fb-4cbb-816f-0fb80c9b5623 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.658273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad33845-69fb-4cbb-816f-0fb80c9b5623 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.658555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad33845-69fb-4cbb-816f-0fb80c9b5623 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.658708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-db749f59-175c-4462-9ee0-1c34685e119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:38.660426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 243/466] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:38.661440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad33845-69fb-4cbb-816f-0fb80c9b5623 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.799613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 224/467] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:38.799613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea341-f658-4932-b34e-f9657501c546 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.799613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea341-f658-4932-b34e-f9657501c546 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.799613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ea341-f658-4932-b34e-f9657501c546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.810534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad33845-69fb-4cbb-816f-0fb80c9b5623 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:38.841966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-780ea341-f658-4932-b34e-f9657501c546 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:38.844360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-780ea341-f658-4932-b34e-f9657501c546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:38.871707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:38.872728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:38.873213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:38.873707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:38.874658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:38.876408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:38.907946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 244/468] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:38 2022] POST /identity/v3/users => generated 246 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:38.909441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:38.932193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:38.977808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 225/469] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:39.021249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ea341-f658-4932-b34e-f9657501c546 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.046451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3aae8-6983-4015-a3f4-9dbac7007b66 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.089299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.090008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.090286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.090557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.102762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.104412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:39.122843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:39.124479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:39.125991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:39.125991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:39.125991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:39.125991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:39.125991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 245/470] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:39.129950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.139886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:39.219118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:39.223632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ea341-f658-4932-b34e-f9657501c546 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.265504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ff8a-3d64-40fc-9c9b-10f5d79b6217 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.288598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.290376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.291166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.291943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.304493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.305502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.305938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.306447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.307328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:39.307796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.309551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.312333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.315378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.324338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 246/471] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:39.330070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:39.348697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:39.349977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:39.351367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:39.351367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:39.351367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:39.351367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:39.355311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 226/472] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:39.365422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.444841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f8df3-949e-4fde-8e41-296a72fb6aaa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.472361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:39.486340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.487021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.487288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.487773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.488741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.490390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fb42c-e74f-4658-b60d-f052e3af8f0c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.515869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 247/473] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:39 2022] POST /identity/v3/users => generated 249 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:01:39.562646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365ac09-068b-45cc-ac10-7ddc03f3bf6b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.578170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:39.617650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837ab0a-a367-4c62-a160-024da519ca27 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.618364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837ab0a-a367-4c62-a160-024da519ca27 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.618888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837ab0a-a367-4c62-a160-024da519ca27 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.620586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 248/474] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:01:39.632179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.633319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.633519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.633797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.634171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.634276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.634818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.635748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:39.636295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.641180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.661476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 227/475] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:39.669995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:39.670766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.671805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:39.804247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 249/476] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:39.830632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:39.862831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b0c7-41f8-4175-8fcf-82281229ea97 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.926843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.928002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.928508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.929158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.930667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.933694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:39.945777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933ed5a-349d-4ad2-9aaf-c87a311fcb52 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:39.959491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:39.974309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 228/477] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:39 2022] POST /identity/v3/users => generated 248 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:39.992976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:39.993799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:39.994088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:39.994366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:39.995165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:39.997532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:40.005983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.sql.core [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 12:01:40.005983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 12:01:40.005983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [parameters: {'id': '160f037a94bd44f591b1a5a5d49392db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 12:01:40.005983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80384) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application result.read() Jul 21 12:01:40.024398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:01:40.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application yield resource Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise self.value Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application session.commit() Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 12:01:40.026303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self.session.flush() Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application compat.raise_( Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 12:01:40.027130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:01:40.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application result.read() Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:01:40.029472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [parameters: {'id': '160f037a94bd44f591b1a5a5d49392db', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:40.030389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 12:01:40.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:40.037830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 250/478] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Jul 21 12:01:39 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:40.037966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:40.084652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:40.212712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad3382-83f8-419d-8ba3-09c1f624563f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:40.278982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:40.280456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:40.280963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:40.281457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:40.301869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-1e98f8bc-6d1a-4c64-9bdc-87829501a16c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:40.318417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:40.320578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:40.322213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 229/479] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:01:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:40.327541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:40.518788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a49c-3506-42fa-82a3-284d2b857454 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:40.586395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:40.588258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:40.589051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:40.589808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:40.591673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:40.592500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:40.595568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09137c82-8fc2-443e-9a62-0636c1311c1b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:40.613216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 251/480] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:41.615923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be873a-8b34-4b5d-a1d1-95000f14e77a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:41.616482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be873a-8b34-4b5d-a1d1-95000f14e77a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:41.616482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be873a-8b34-4b5d-a1d1-95000f14e77a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:41.618080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 230/481] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:41.628564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:41.629554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:41.630050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:41.656824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:41.658530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:41.699440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:41.764533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 252/482] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:41.787467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be873a-8b34-4b5d-a1d1-95000f14e77a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:41.876192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:41.908304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be873a-8b34-4b5d-a1d1-95000f14e77a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:41.958780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:41.959652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:41.959941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:41.960219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:41.980548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-e4fc2b52-68af-4e93-b4a8-3dbb97d17456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:41.988512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:41.989511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:41.990315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 231/483] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:41.996100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.133990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f3fd-fcbb-4d74-b2f2-0da7208e2cec None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.134714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f3fd-fcbb-4d74-b2f2-0da7208e2cec None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.135201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f3fd-fcbb-4d74-b2f2-0da7208e2cec None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.137200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 232/484] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:42.143304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.143969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.144457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.156514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df422dc0-986c-4b7e-94bf-d5ff575095cc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:42.185077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:42.186295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:42.235215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:42.235943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.236225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.236498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.237285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:42.237591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:42.239096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:42.256239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 253/485] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:42.260536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.322940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:42.350627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 233/486] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:01:42.382294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.452323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d0de-449a-40ee-ba08-317fa06b3472 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:42.530477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:42.531302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.531606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.531866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.531964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:42.553086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:42.554613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:42.559979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650e22a-b3a4-48d1-988c-5a60c0e562be None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:42.596293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:42.597241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:42.598046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:42.598046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:42.598046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:42.598046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:42.598046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 254/487] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:42.602425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:42.603670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.603799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.604285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.604791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.617147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:42.625944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:42.626892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:42.627726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:42.630522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 234/488] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:42.636834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.753340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e059a06-7d4f-4ca6-b7ec-c201d7db8873 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:42.757068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:42.788966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:42.795930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c2016-b2e1-4846-9faf-746e3a035d46 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:42.796412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:42.797385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.797890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.798387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.799403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:42.799961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:42.801637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e140e0-049f-4129-b4d4-4bd4c7cea537 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:42.812185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 235/489] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:42.818549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299961d-79fc-44d6-91ed-4b72cfb78700 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.819163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299961d-79fc-44d6-91ed-4b72cfb78700 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.819679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299961d-79fc-44d6-91ed-4b72cfb78700 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.821702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 236/490] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:42.822854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299961d-79fc-44d6-91ed-4b72cfb78700 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:42.856886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:42.858260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.858750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.859881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.860887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:42.861689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:42.863360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf76b7-1097-48ed-9de8-0243f9d2fdf2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:42.880653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 255/491] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:42.882761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:42.883882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:42.884497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:42.911832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:42.913575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:43.004947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 256/492] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:43.006972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.064286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299961d-79fc-44d6-91ed-4b72cfb78700 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.121013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.132579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12cacc-5a7e-4c1a-afeb-33e52550fcad None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.137202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.138677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.139168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.139713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.162333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.165339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.182153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.184071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.184591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.185352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.194043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.195932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:43.202576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:43.203448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:43.204324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:43.204324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:43.204324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:43.204324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:43.204947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 237/493] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:43.206268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:43.218338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:43.219303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:43.220229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:43.222968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 257/494] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:43.225071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.297600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.354993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e663ed-b642-4542-8f73-00c1196e8556 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.385693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.392523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.393212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.393496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.393768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.394514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:43.394821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.396298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.397166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acca363-4837-4301-80d9-07f2829f198f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.409379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 258/495] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:43.410264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.450548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.451389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.451707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.451986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.465812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-0202fb15-3848-4cdc-ac4f-248f756879c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:43.473715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:43.475229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 238/496] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:43.477321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d34c-fccb-4de3-acf7-115760d26562 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.478760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d34c-fccb-4de3-acf7-115760d26562 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.478760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d34c-fccb-4de3-acf7-115760d26562 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.483742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 239/497] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:43.484744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1d34c-fccb-4de3-acf7-115760d26562 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.554683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55273c82-eeaa-40f2-ae36-bab0d7414e07 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.604954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.605712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.606127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.606483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.607437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:43.607888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.609780 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.619552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1d34c-fccb-4de3-acf7-115760d26562 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.627969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.652309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 259/498] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:43.653217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.669258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.670115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.670395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.670674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.679653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.681185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-8f92fb7d-c339-4a01-a78a-9c7827926a7c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:43.700190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:43.702005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 240/499] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba70-7fa0-42f0-bf16-4a85dd313580 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.703871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba70-7fa0-42f0-bf16-4a85dd313580 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.705883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4ba70-7fa0-42f0-bf16-4a85dd313580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.722078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fff4ba70-7fa0-42f0-bf16-4a85dd313580 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:43.723334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fff4ba70-7fa0-42f0-bf16-4a85dd313580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:43.726155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.788683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd5c5c-b9e6-4464-9520-ba21965ec8bf None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.808626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 241/500] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:43.809937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5790-6dc9-47cc-98b4-6753dbfaa7ae None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.810221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5790-6dc9-47cc-98b4-6753dbfaa7ae None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.810485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5790-6dc9-47cc-98b4-6753dbfaa7ae None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.811812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 242/501] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:43.812924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c5790-6dc9-47cc-98b4-6753dbfaa7ae None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.838969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.839697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.839983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.840262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:43.841032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:43.841343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:43.842767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:43.852682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 260/502] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:43.853633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:43.928990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.937570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c5790-6dc9-47cc-98b4-6753dbfaa7ae None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.968108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37aaef-0e99-4417-91d5-36664c643ee7 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:43.979031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:43.984013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:43.985147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:43.985635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:43.986089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.003798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.004456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.004744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.005015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.005902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:44.006211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.007695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40ed71-4cfb-4ac1-b7a8-ac2ba0217e3e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.028122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 261/503] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:44.029578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.029871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.030144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.053457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:44.054783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:44.095208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.100492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.135521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 262/504] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:44.136373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.227711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.243522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 243/505] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:44.243522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.255378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.260484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388edc94-f548-4fa0-be5d-1de3d7807a1a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.300695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.301509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.301783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.302052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.313136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:44.320863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:44.322578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 263/506] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:44.323415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.366925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9994cc-77aa-42bf-b7b3-764c6d8e1e85 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.407734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.408786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.409066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.409335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.421662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.423140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:44.443519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:44.445356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 244/507] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:44.447251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.462557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43abf23-5f0a-4727-99e8-1b0d368afbf0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.511029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.512103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.512561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.512936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.521619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.554474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.592102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5dfef-eaa1-477f-b6c2-66a473be13ae None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.609590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.611181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.636853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.637898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.638359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.638734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.651509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.653142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:44.672250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:44.674072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 245/508] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:44.675756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.745658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 264/509] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:44.746889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.805897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.810116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83141e28-908a-4ee2-ad6c-0f5f99cdcb57 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.827215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:44.857563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.858730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.859192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.859636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.872487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.874165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.883689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783e0f-2d97-4d07-8af1-448d3da975e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:44.901838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:44.903626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:44.905412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:44.905412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:44.905412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:44.905412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:44.905412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 246/510] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:44.906039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:44.922286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:44.922969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:44.923250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:44.923557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:44.924350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:44.924651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:44.926073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:44.934852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 265/511] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:44.935735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.051702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005bac1-ea82-4f9f-ae63-a4a7fddca1e1 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.057327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69730630-24af-4673-b066-45b9591e57e2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.075536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.097994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.099076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.099584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.100054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.101370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:45.101760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.101907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.102803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.103224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.103715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.104383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.105013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:45.105502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.107967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.125868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 266/512] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:45.127347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.127930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 247/513] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:45.129234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.152526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.271096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef2c0c-3c1b-465a-8420-fd37a79f7bc8 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.273128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d163d-e4e0-4f11-aa75-1ab3b3909766 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.311699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.312666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.313047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.313383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.315058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.315952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.316331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.316759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.317608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:45.318004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.319512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.325153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.326616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.333443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 248/514] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:45.334302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:45.353844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:45.355733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:45.357528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:45.357528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:45.357528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:45.357528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:45.359722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 267/515] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:45.361305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.400816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.414636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.446358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73861bf-6d7e-4e48-b633-3cf8aa46ee9a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.483968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.484892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.485270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.485624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.497711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.499516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.500686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865205a1-b497-41b9-81ca-5aa5d7f0badb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:45.520249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:45.522136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 249/516] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:45.523884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.553812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.554951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.555427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.555967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.557299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.559869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.589619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 268/517] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:01:45 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:01:45.590488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.667351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73fb56-0cdc-4ad5-bc08-be85701a7d28 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.703667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.712046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:45.722304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.723398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.723837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.724164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.733246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8b1b7-32da-44fc-9b8e-7c3b1290543b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.734018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.735769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:45.754335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:45.755372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:01:45.756290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:45.756781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 250/518] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:45.758745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.787317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.788298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.788737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.789799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.799818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.802137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:45.823423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:45.825099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 269/519] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:45.828992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:45.919693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13486a-5b06-4117-814e-8c2d7d91c33c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.983250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:45.984421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:45.984923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:45.985400 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:45.986745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:45.987278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:45.987930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2efa-1126-473b-8d27-09277e4237c5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:45.989911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.016834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 251/520] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:46.016834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.039959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.040750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.041025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.041344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.042096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:46.042399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.043997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.058211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 270/521] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:46.058833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.076417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.097885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.161841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6ef5d-c2bc-4f1a-a7e6-90f623d19da7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.200295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.200876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.201267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.201415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.202261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:46.202570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.204132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.245212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 252/522] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:46.245428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.263519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a82e-31c0-422d-81f4-7f0ec6d2f2c2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.317883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.318728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.319009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.319282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.320452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:46.320758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.322367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.349847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 271/523] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:46.350564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.387837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192dcc6-d3f9-490a-a2ed-ba5408804345 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.389904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.425490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.426313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.426591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.426860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.436677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.438363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:46.453804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:46.455322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:46.456941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:46.458156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 253/524] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:46.459684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.481920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.521704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43aecd3-d19e-42bc-8cd8-d7b60c146cb5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.570311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.571806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.572711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.572711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.584763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.587702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:46.612163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:46.613512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:01:46.614609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:46.615225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8bc5f-1fc1-4f6a-9895-29171295d0b3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.621395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 272/525] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:46.621395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.656422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.657087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.657367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.657645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.689378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.744998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.746537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.763639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616fec3-cde4-4922-8770-11e5e2627633 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:46.794959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.806702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:46.807836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:46.808330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:46.808817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:46.810089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:46.811297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:46.814872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:46.825002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 273/526] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:46.837534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.890168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 254/527] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:46.891445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:46.993228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:46.999322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ed56-df9b-437e-a6e0-677b10d64507 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.045892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.046681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.046965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.047246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.048111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:47.048427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.049930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.062402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87988963-b161-4947-88ba-71835a8a8659 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.069078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 274/528] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:47.069078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:47.126748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.127813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.128282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.128776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.129819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:47.130489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.133852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.150055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:47.165084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 255/529] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:01:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:47.166389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:47.251467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046732e-1f11-4086-a82e-b99115a1027f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.294073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c210b5-c8e2-4402-8acd-6d82110c5701 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.306108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.307915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.308679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.309418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.318605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:47.332141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.335854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.338945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.340045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.340394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.340732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.341647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=f54544af7d84443e9b8fdbcb718bf824) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:47.342028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.344083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.359518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 256/530] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:47 2022] GET /identity/v3/role_assignments?user.id=f54544af7d84443e9b8fdbcb718bf824&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:47.359518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:47.377120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:47.378042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:47.378868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:47.378868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:47.378868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:47.378868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:47.388042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 275/531] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:47.393445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:47.553939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d88404-3806-453f-9e5a-1528cf80228e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.617063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b609c5-daee-4ef7-93c4-7a53921b8792 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:47.635445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:47.636431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.636937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.637683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.637683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.730113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:47.738906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.740679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f2b52-46ab-479e-aff6-e06fb0f87590 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.750759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:47.751458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.752290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.752568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.753314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:47.753622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:47.757128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:47.781467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 276/532] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:47.786092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:47.976493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 257/533] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:47.977713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd805ca-418a-45c4-bcd5-86fcc60f7603 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.978045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd805ca-418a-45c4-bcd5-86fcc60f7603 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.978325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd805ca-418a-45c4-bcd5-86fcc60f7603 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:47.984028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 258/534] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:47.985418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:47.985708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:47.985988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.003842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:48.005136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:48.025135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9c0f3-852a-4cf5-804c-329b6802d183 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.075169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.076370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.076655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.076925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.087989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.089854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.102733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 259/535] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:48.103914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:48.109139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:48.110001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:48.110806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 277/536] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:48.116052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.136820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.158378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.247281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9163-240e-450f-b5f1-1f524bd49ddc None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.268907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63cbe3-5149-43fa-a6a9-3c0d51b20f1f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.286436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.287284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.287570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.287844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.301054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.302558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.321842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.322748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.323164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.323617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.324574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:48.325015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.326613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:48.332551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:48.333661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:48.334642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:48.334642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:48.334642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:48.334642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:48.335726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 260/537] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:48.337335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.348026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 278/538] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:48.349130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.451872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.460986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.476961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac58485-7ff0-4a0e-bcf3-bfc328e40742 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.518862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.519813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.520230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.520660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.521533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:48.521843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b5cf9-0041-4c53-a199-3c6e5ae83a5c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.522138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.523718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.540279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 279/539] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:48.541986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.585029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.587158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.587708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.588433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.604790 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:48.614743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:48.615940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:48.616873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:48.619947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 261/540] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:48.622079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.691142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5136-cf0f-483a-a1bc-5f07fcf1bc0d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.732911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658969cb-0766-492e-8228-53a138402980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.733755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658969cb-0766-492e-8228-53a138402980 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.734164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658969cb-0766-492e-8228-53a138402980 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.734541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658969cb-0766-492e-8228-53a138402980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.735565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658969cb-0766-492e-8228-53a138402980 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=fcb115c5cad84f55b143c392c8c23dad) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:48.735994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658969cb-0766-492e-8228-53a138402980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.737624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658969cb-0766-492e-8228-53a138402980 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.744666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 280/541] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/role_assignments?user.id=fcb115c5cad84f55b143c392c8c23dad&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:48.745735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658969cb-0766-492e-8228-53a138402980 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.758130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.857330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f94f0d-599d-4b66-bf0c-5e48d2982f33 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.867624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658969cb-0766-492e-8228-53a138402980 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:48.908468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.909562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.910043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.910517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.911549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:48.912086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.913791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:48.919905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:48.924675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 281/542] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:48.934433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:48.948408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:48.949271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:48.949553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:48.949820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:48.968535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:48.970124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:49.005241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:49.006753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:49.007955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 262/543] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:49.012315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.069718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:49.107964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54230d9e-9a3e-4300-95aa-9fe7506096ac None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.162890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.164171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.164635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.165280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.180609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:49.183203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1ea29968-5c99-4a38-9d99-b646ad06b105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:49.206663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:49.207959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:49.209007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:49.209007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:49.209007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:49.209007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:49.211319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4e407-1b23-4a39-a389-d61d604fe365 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.217770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 282/544] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:49.220075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135fdf-8394-4021-bdc0-48e68882b3ac None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.220638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135fdf-8394-4021-bdc0-48e68882b3ac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.221125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87135fdf-8394-4021-bdc0-48e68882b3ac None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.223357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 283/545] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:49.224539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87135fdf-8394-4021-bdc0-48e68882b3ac None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.266883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.267960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.268438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.268932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.270503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:49.271061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:49.273762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf1d48-8e09-4042-9151-f47e475128e9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.317553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 263/546] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:49.320230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.320644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.321190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.341807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87135fdf-8394-4021-bdc0-48e68882b3ac None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.354359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:49.356230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:49.368170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:49.381249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.382410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.382913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.383386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.384459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:49.384972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:49.386690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.400527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 284/547] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:49.401774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.443856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:49.493164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 264/548] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:49.495374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.520516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8ceb3-fd2e-472c-832d-8331944f933d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.561576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.562315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.562696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.563075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.564061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=5493d9cde9404225bedc423f0ecf213c) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:49.564497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:49.566138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.579531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 285/549] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/role_assignments?user.id=5493d9cde9404225bedc423f0ecf213c&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:49.579531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.654724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:49.710301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60024cc2-2142-4c16-b966-8e469ac57b12 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.751174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.753286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.754365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.754668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.763594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:49.765124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:49.766248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033897f-8db0-4c89-8c49-f9a75a5a58db None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-f2862504-f223-4d9c-9d1f-31e3246a313b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:49.782789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:49.784982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 265/550] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2435f-9cde-412d-ac32-47b3228b871e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.787093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2435f-9cde-412d-ac32-47b3228b871e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.795334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2435f-9cde-412d-ac32-47b3228b871e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.795334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 266/551] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:49.795334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2435f-9cde-412d-ac32-47b3228b871e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:49.842716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:49.848383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:49.850597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.851731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.852497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.874895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-3c16b9ca-2f58-4778-9064-ce8e922cd72f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:49.897807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:49.899866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:49.902525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 286/552] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:49.904915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:49.904915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:49.904915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:49.928659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2435f-9cde-412d-ac32-47b3228b871e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:49.929446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:49.929951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:50.007578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.009220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.009911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.010948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.012516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:50.013287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.016507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.039052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 287/553] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:50.039052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.052679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 267/554] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:01:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:50.062693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.127891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.196269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.202632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6936b-6010-45b9-a86b-a7f5548306ed None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.237485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964714-27e2-42fe-875f-c0f49e899530 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.267581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.269678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.270303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.270925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.291715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:50.299992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.301271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.301905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.302514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.304008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:50.304572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.307103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:50.310459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:50.311381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:50.312227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:50.320132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 268/555] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:50.321415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.326376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 288/556] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:50.326376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.454406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc559d9-f030-4e97-919a-49e6e78a16ad None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.495511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.496733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.497227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.497772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.517416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.523585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdb29f-4172-4767-8ea8-359e6f3d6b98 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.539899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.583543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.584915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.586076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.586564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.587138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.587818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.589323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:50.589930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.592525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.612765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 289/557] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:50.612765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.708542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 269/558] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:01:50.709776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.775955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.805284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03387ab4-2260-4dfa-baf5-14fe0806905c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.841561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de52a2e-715b-44e6-8ad8-114012892d66 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:50.854923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.855997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.856293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.856564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.871373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.873515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:50.895528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:50.897225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:50.898972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:50.898972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:50.898972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:50.898972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:50.901623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 290/559] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:50.903278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:50.916747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:50.918852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:50.919735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:50.920668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:50.924696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:50.944970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:50.949093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:50.980243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:50.981260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:50.982118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:50.982118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:50.982118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:50.982118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:50.985590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 270/560] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:50.987292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.041116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa718-5d4f-4385-9cb2-d35bd18ecc71 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.087678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.088557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.088930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.089425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.105046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.106581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.126703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e8f2e-2774-400c-8e4f-e147296262a1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.135831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:51.141121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:51.142400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:51.143405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:51.143405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:51.143405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:51.143405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:51.150077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 291/561] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:51.150077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.178718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.179555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.180096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.180293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.181146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:51.181522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.183023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5576a-ea2f-432e-99e3-0aa2a0b98472 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.196912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 271/562] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:51.198462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d97901-bc71-46c8-9d74-a9e234032468 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.198757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d97901-bc71-46c8-9d74-a9e234032468 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.199037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d97901-bc71-46c8-9d74-a9e234032468 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.204697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 272/563] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:51.204697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d97901-bc71-46c8-9d74-a9e234032468 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.249053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:51.308878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73869f41-7400-4e75-b975-2a082dd0ab6f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.333011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d97901-bc71-46c8-9d74-a9e234032468 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.351297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.352003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.352291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.352567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.354799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:51.354799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.355071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.369590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 292/564] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:51.375550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.389000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.389695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.394246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.394246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.394246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:51.394246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.394246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9e464-a67f-4b62-82bc-0a03e8c6c214 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.409551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 273/565] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:51.410797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.411101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.411344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.430934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:51.432207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:51.457591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:51.521879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a32b5-935f-46e3-8ccc-92d0092e9264 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.541388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 274/566] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:51.543012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.567326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:51.587855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.589263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.589761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.590249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.604335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.607011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:51.631549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:51.633401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 293/567] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:51.635467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.737541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb31fa-72c8-452a-98a8-7a89d2ccbedc None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.787558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.788380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.788664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.788934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.798088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.799646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:51.816283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:51.818021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:51.819651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:51.822773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 275/568] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:51.823768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.829101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc35ae6-f9b6-4f83-a5b1-48b912469700 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.847008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:51.891211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:51.892069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:51.892346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:51.892603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:51.900287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:51.901772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:51.916691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:51.918541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 294/569] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:51.920324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:51.962199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8184-0dec-458b-97e4-e2038adc3f94 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:51.974715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.015307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.016398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.016784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.017133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.030480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:52.036827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76912f3-2a9d-4611-bdba-6f5aa51fee5a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:52.039021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:52.041081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 276/570] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:52.043106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.082302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.082997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.083275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.083578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.084465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:52.084765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.086666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.106811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 295/571] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:01:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:52.107731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.133316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.164506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c2d0a-df31-4bcb-b13a-c99c8e407803 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.204765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.205481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.205757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.206018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.206900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:52.207208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.208719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28faae-2093-4f33-ab0d-a216c774215c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.228826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 277/572] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:52.229997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9573b848-f3bb-4e1d-a2fb-7c88d9d59dff None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.230718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719a66-78bb-4db0-b992-a22c64ea80e7 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.230718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719a66-78bb-4db0-b992-a22c64ea80e7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.230718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719a66-78bb-4db0-b992-a22c64ea80e7 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.232424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.232595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 278/573] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:52.233078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719a66-78bb-4db0-b992-a22c64ea80e7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.274070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.274822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.275079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.275345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.276175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:52.276481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.277946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.291014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 296/574] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:52.291818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.354014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719a66-78bb-4db0-b992-a22c64ea80e7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.389782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b592-1944-47dd-af29-42f47979810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.390470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b592-1944-47dd-af29-42f47979810a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.390754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b592-1944-47dd-af29-42f47979810a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.391029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805b592-1944-47dd-af29-42f47979810a None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.391843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805b592-1944-47dd-af29-42f47979810a None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:52.392151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805b592-1944-47dd-af29-42f47979810a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.393596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805b592-1944-47dd-af29-42f47979810a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.395307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.399940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c437e6-d600-40eb-9c80-7de86f9d5384 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.405063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 279/575] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:52.405933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b592-1944-47dd-af29-42f47979810a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.439783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.440589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.440936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.441215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.452699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.454194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ce1734d8-c805-4f96-9265-7170cf45e745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:52.472525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:52.473528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 297/576] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.475665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.493056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:52.494371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:52.505344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.518214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805b592-1944-47dd-af29-42f47979810a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.555186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.555993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.556307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.556577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.568379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.568835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 298/577] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:52.569710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.569863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:52.596357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:52.597940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 280/578] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:52.599744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.679147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.686839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3abcd-5395-434e-8c7c-658a5cd5a8de None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.740892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.741680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.742047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.742386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.743305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=d14dd264bd4343b49ef968789de08f73) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:52.743795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.745379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.745660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e97aa-bb04-496b-9976-ec0ee77e1902 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:52.752055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 299/579] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/role_assignments?user.id=d14dd264bd4343b49ef968789de08f73&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:52.753009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.771838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:52.804140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:52.805468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:52.805759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:52.806202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:52.820414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:52.822111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:52.844915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:52.846586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:52.848210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:52.848210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:52.848210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:52.848210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:52.848210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 281/580] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:52.849163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:52.935460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ffa-4447-47eb-9558-3b29ddea9a4b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.002373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.003578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.004065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.004762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.006100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:53.006630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.009309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.015414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:53.020173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107144d-3b88-4d3c-a0d0-aa4b977c62be None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.042569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 300/581] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:53.042569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.065944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.066692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.066971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.067309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.081404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:53.090154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:53.092365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 282/582] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:53.094194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.159124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:53.218270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47ef0d-7f2d-4721-9646-b8442f8dcae6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.272523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3375-d665-4662-87d8-4e3678aa23bf None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.282840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.284447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.284908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.285343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.287106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:53.287670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.290260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.318421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 301/583] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:53.318421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.323070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.324270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.324797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.325195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.326134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:53.326713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.329144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.350539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 283/584] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:53.352111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.390408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:53.505298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2a1e8-09c7-4266-8867-73e74a00ebd9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.526598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2362dc-8d93-4d97-96d7-f0924bb008ec None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.564501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:53.574119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.574956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.575386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.575762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.576624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:53.577030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.578640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.581260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.583120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.583941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.584631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.590382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 284/585] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:53.591940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.606298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.609509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:53.638857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:53.640665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:53.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 302/586] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:53.644564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.723620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:53.754508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e29dc9-5609-4e2b-9c8b-c0244fddbb22 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.807192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.808302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.808747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.809309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.817501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626775-6b05-423f-95e6-4cc2bb65fabe None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:53.820511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.822472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:53.849095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:53.850850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 285/587] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:53.861986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:53.885664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:53.887092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:53.887677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:53.888240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:53.901591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:53.903811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:53.928746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:53.930840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:01:53.932972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:53.935399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 303/588] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:53.937410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.000124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.041227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7dda-6643-4ed1-9253-dba4fb45d2a7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.102259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc2b7-c137-47c7-aac8-f6563c4f275a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.103680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.104925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.105482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.106045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.122904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.125344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.128810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:54.155818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:54.157753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:54.159703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:54.159703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:54.159703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:54.159703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:54.166489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 286/589] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:54.167999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.172575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.174537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.174537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.174537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.175660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:54.176247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.178853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.206008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 304/590] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:54.207474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.358175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edad550-94ca-414d-bf75-f9b5d5343b3f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.360990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664064-c55b-48cb-944b-6f88714b659e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.377343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.424502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.424730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.425630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.425767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.426125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.426223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.426613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.426707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.428213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:54.428313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:54.428777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.428874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.431632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.431760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.459739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 287/591] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:54.459739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.472825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 305/592] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Jul 21 12:01:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:54.472825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.548606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.609951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04883e-b40f-465a-9d77-8f42d9b49807 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.640890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed9b30-823f-413d-ba24-6714a13cefd0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.657653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.658822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.659372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.660216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.661567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:54.662156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.664604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.690946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 288/593] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:54.690946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.701948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.702949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.703237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.703527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.733160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=bccc158b75eb41ca8431bbb67af65260, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.735987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:54.759037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 306/594] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:54 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:54.769904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:54.797477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.890176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a5689-a637-49b7-b5c3-5b1869a27c9b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.924366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:54.963408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242db24-b55c-44c7-9741-969d2976bf0a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:54.970026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:54.970960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:54.971243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:54.971549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/f54544af7d84443e9b8fdbcb718bf824/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:54.996085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=f54544af7d84443e9b8fdbcb718bf824, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:54.997597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.016429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 289/595] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:54 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/f54544af7d84443e9b8fdbcb718bf824/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:55.017894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.039012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.040217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.040671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.041125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.060524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.062356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b7607c27-930c-4d92-82e5-28d5d5417db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:55.091038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:55.092048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:55.092902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:55.092902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:55.092902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:55.092902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:55.099276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 307/596] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:55.099276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c180c5-3123-4371-9171-29793f0be00a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.099276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c180c5-3123-4371-9171-29793f0be00a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.099276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c180c5-3123-4371-9171-29793f0be00a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.105505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 308/597] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:55.107002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c180c5-3123-4371-9171-29793f0be00a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.198295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e174-b6d0-42ad-9a6d-7a149fb0ba44 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.226499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:55.244887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.246145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.246753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.247241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.256680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.258616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2368b75e-452e-45fb-8fa8-33947a146210 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:55.276294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:55.277283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:01:55.278202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:55.283757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 290/598] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:55.285583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.286027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.286445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.291739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c180c5-3123-4371-9171-29793f0be00a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.308786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:55.310155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:55.339877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:55.355331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.357523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.357893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.358243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.359064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:55.359437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.360950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.379591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 309/599] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:55.380512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.402215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 291/600] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:55.403161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.531132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f47b4-2dd9-4440-afb2-c9fb4caa1d03 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.556486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a2b79-485f-4ba5-8705-e51d144fd71c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.567395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:55.580825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.582044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.582549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.583166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.584754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:55.585719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.588768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.605396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.606581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.607065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.607578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.616968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 310/601] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:55.616968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.621156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.623814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:55.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:55.648600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:55.650039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:55.655832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 292/602] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:55.657260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.679909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:55.747851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0200cce-fa55-44e9-af38-c4f2c52254dc None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.784286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8b344-2f95-4281-ab98-9ef8737f421b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:55.790595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.791974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.792486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.792980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.806257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:55.814904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:55.816192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:55.817021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:55.823853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 311/603] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:55.825434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.839643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:55.842056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:55.842619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:55.843812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:55.870819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:55.874298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:55.907385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:55.909166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:55.910947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:55.910947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:55.910947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:55.910947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:55.910947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 293/604] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:55.912274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:55.913323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.026383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f744e-4d4b-444c-8790-8bb5d5af8c96 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.031432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.078418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356349d-3364-40ab-8e67-a36b75f04f67 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.098617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.100438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.101204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.101964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.104230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:56.105106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.108483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.114708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.115398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.115724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.115980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.116932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:56.117243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.118706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.135561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 294/605] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:56.135561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.145213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 312/606] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:56.148061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.236215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cb5f0-f90f-4451-99da-bc70d138d1a7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.264027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.273792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.274893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.274893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.275167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.275991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:56.276302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.277723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.291541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 295/607] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:56.292253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.362132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58549679-4bd3-4c63-b988-9f0a001d89cc None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.379957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.406247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711215e0-7092-4fc2-ad13-f8e2c86d519d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.436122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.438258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.439073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.439914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.443007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.444096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.444277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.444563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/fcb115c5cad84f55b143c392c8c23dad/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.461670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=fcb115c5cad84f55b143c392c8c23dad, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.462920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.463220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.466634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.481289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 296/608] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:56 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/fcb115c5cad84f55b143c392c8c23dad/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:56.482085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:56.504559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:56.506330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:56.507997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:56.507997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:56.507997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:56.507997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:56.512492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 313/609] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:56.522492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.530211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.603187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a6495-4d33-44bb-b0db-8dd8fbd23777 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.643509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.644275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.644555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.644827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.653527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.655598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:56.672345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:56.674140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:56.675963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 297/610] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:56.679807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.770292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdda4e-832c-4f0f-b0b4-f71414bf2b10 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.842707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.843405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.843752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.844013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.844771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:56.845080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.846521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.855922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76b570-c790-4de5-a2d6-18b76fda5627 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:56.869947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.882047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 314/611] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:56.892323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:56.897467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:56.898274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.898553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.898825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.899909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:56.900219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:56.901708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbc9f3-1895-4784-9c3a-21dda654f10e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:56.919668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 298/612] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:56.919668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da9fd-10d9-41d6-b56c-451d8bd8fe92 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:56.919668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da9fd-10d9-41d6-b56c-451d8bd8fe92 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:56.919668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da9fd-10d9-41d6-b56c-451d8bd8fe92 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:56.919668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 299/613] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:56.923318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:56.925978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546da9fd-10d9-41d6-b56c-451d8bd8fe92 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.028953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40065b76-8971-4f3f-b282-86ac9882b867 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.066160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.066977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.067299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.067560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.076577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:57.078186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:57.084903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546da9fd-10d9-41d6-b56c-451d8bd8fe92 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-2cc2fbb6-c546-41b4-a623-f0508ad08bcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:57.093888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:57.096266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:01:57.098060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:57.100267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 315/614] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:01:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:57.101781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.102064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.102340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.120428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:57.121765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:57.159601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.160576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.160854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.161134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/5493d9cde9404225bedc423f0ecf213c/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.177615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=5493d9cde9404225bedc423f0ecf213c, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:57.179211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:57.196245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 300/615] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:56 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/5493d9cde9404225bedc423f0ecf213c/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:57.196702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 316/616] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:57.197248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.226789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:57.241188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.264856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:57.374697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a015c0-30e9-4055-9b8e-14cf663754c3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.417187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.418217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.418506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.418783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.455868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:57.465474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec189720-5ef8-47f8-974f-99e0656e5c70 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:57.469375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:57.471374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:57.473288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:57.477313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 317/617] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:57.483542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.509832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.510534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.511334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.511643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.512920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:57.513235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:57.514931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:57.538309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 301/618] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:57.544935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.608092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:57.681231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94f5e-254b-422f-aa61-92f4ef679195 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.686342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:57.723887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.724568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.724846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.725210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.725868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:57.726282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:57.727767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41e89c-11c4-458d-bab9-208de38bc914 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:57.737627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 318/619] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:01:57.739336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830d5e-18d0-4ca9-a261-bdc94c4f8549 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.739535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830d5e-18d0-4ca9-a261-bdc94c4f8549 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.739815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830d5e-18d0-4ca9-a261-bdc94c4f8549 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.746125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 319/620] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:57.746125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41830d5e-18d0-4ca9-a261-bdc94c4f8549 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.765674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcd2cb-ff46-477c-b032-f491e1884ecd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.817979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.819071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.819539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.819957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.831990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:57.833755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-cb93e89a-85d3-4109-a89b-2f037503e6f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:57.853901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:57.854865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:57.855781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:57.855781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:57.855781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:57.855781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:57.857747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 302/621] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:01:57.859293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.859759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.860175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:57.878702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:57.880271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:57.910654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41830d5e-18d0-4ca9-a261-bdc94c4f8549 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:57.957552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 303/622] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:57.964688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:57.968690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:57.987755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:57.989770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:57.990447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:57.991892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.027156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.029920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:58.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:58.072713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:58.074726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:58.074726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:58.074726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:58.074726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:58.077722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 320/623] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:58.079340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.091863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:58.190776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e2dd6-9f45-44f0-9de0-a70a97f6fe21 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.257055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.258597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.259397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.260041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.261422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:58.262206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.264360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.292007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 304/624] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:58.293928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.317801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce1d33-da2e-449d-ac42-fff9f3e8a663 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.406453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.408347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.408985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.409626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.436349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:01:58.448990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:58.450288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:01:58.451262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:58.455201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 321/625] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:58.456977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.486371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f83d-ded8-4730-a1c5-b58eec18312f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.503660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:58.553250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:58.553973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.556807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.559692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.560366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.561728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:58.564098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.566487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.597837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 305/626] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:58.600115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.677556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15b318-12d4-45a7-9452-4f6c7e66c35d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.721131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5157729-2e32-4b26-b867-22341cc85b53 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.738338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.740106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.740846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.741470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.752281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.755008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.759470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.760283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.760568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.760853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.761673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:58.762009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.763474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.777658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 306/627] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:58.778628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:01:58.786113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:58.787247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:01:58.788341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:58.792032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 322/628] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:58.793868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:58.814299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:58.895454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8fe68-1884-465d-be24-e68ec7081cee None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.936840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:58.937668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:58.937951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:58.938230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:58.947781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:58.949419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:58.954420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c81df-3b7c-431e-98f7-380b3343f94f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:58.957744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:58.968083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:58.969728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:01:58.971349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:01:58.974496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 307/629] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:58.975887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.023270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.025142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.025972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.026737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.046374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.048947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:01:59.079861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:59.081179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:59.082555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:01:59.082555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:01:59.082555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:01:59.082555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:59.082555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 323/630] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:59.084106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.089226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:59.102864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca8401-4101-4fbb-9c2b-2af8c813405f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.163357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.164532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.165029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.165654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.166967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:59.167586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.170219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47e2b-23e7-4d57-9427-eacc5180b660 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.201170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 308/631] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:01:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:59.201170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb610b9-b09c-4172-8c3a-8beec50b1a52 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.201170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb610b9-b09c-4172-8c3a-8beec50b1a52 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.201170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb610b9-b09c-4172-8c3a-8beec50b1a52 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.201170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 309/632] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:01:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:01:59.202971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb610b9-b09c-4172-8c3a-8beec50b1a52 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.269501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc781b-8283-40c3-b860-f78ccee8f4b3 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.329514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.330264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.330531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.330800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.331631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:59.331938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.334005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.357740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 324/633] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:01:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:59.363726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.376343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb610b9-b09c-4172-8c3a-8beec50b1a52 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.383838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:59.433696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.434904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.435399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.435931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.437322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:01:59.437877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.440471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cec615-a447-4d1b-967f-527c0e06ff09 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.466356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 310/634] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:01:59.466356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.466356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.466356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.492115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:01:59.494628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:01:59.500779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:59.532600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ac82-a2c2-4759-bb0a-6a887e4a6c96 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.593647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.595019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.595555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.596049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/d14dd264bd4343b49ef968789de08f73/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.615237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 311/635] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:01:59.617101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.633280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=d14dd264bd4343b49ef968789de08f73, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.636122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.660923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 325/636] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:01:59 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/d14dd264bd4343b49ef968789de08f73/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:01:59.663615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:01:59.809196 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:59.810203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114aa97-92bc-4aa9-9b31-35347a01a0d3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.886657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.888123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.888704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5efc7-4fac-49c5-ab79-ab6198550064 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:01:59.888960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.889579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.915133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:01:59.962734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:01:59.964041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:01:59.964466 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:01:59.964886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:01:59.975918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:01:59.978366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:01:59.996394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:01:59.997426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:01:59.998412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 326/637] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:01:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:01:59.999034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.032784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.034350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97644c4d-a4f9-404c-896a-0c4166bd4ac6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:00.159027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f946f-919a-43a3-842d-0b30d95c04e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.186945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.203314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 312/638] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:01:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:00.205469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ebd63-6d4b-40f0-8f3c-95b1104b0882 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.205963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ebd63-6d4b-40f0-8f3c-95b1104b0882 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.206446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ebd63-6d4b-40f0-8f3c-95b1104b0882 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.210797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.211684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 313/639] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:00.212938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.213370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ebd63-6d4b-40f0-8f3c-95b1104b0882 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.213930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.214903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.234030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:00.247009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:00.248915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 327/640] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:00.250524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.308335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.393266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ebd63-6d4b-40f0-8f3c-95b1104b0882 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.434713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.435518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.435966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.436371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.437634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:00.438061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.440000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c94d41-d661-4874-a441-a0215d029de2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:00.448470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e036de-5302-47b4-8a6f-75a11789e735 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.472252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 314/641] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:00.472252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.472252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.472252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.490713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:00.491966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:00.521413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.522734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.523163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.523628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.542806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.545471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:00.570737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:00.572670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 328/642] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:00.575078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.595868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 315/643] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:00.595868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.633031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.651853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.741929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb316f32-babe-4985-a033-30aafdf1ba5f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.762485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5637-458e-4f45-a66f-750a5969e74f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.792881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.793575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.794033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.794342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.795089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:00.795394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.796862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:00.807000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 329/644] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:00.807927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:00.819032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.820347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.820841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.821671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.932703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.939646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504f69f-715a-4125-aad5-3890db3e56ac None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:00.945635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.947180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:00.982589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:00.989494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:00.990664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:00.991161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:00.991700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:00.993155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:00.993712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:00.996553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.019175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 330/645] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:01.020747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.109832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 316/646] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:01.109832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.152756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a8d3-f687-4c6e-ab1f-e54f38036e99 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.206755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.208062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.208512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.208961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.227088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:01.240810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:01.241842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:01.242827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:01.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 331/647] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:01.244544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.267322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83f5b-8dc3-434c-924c-d536016b830e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.285610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:01.315919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:01.323827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.324267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.324614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.324938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.338971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:01.340514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:01.363456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:01.365282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 317/648] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:01.367000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.432863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573c1ce-e926-4a12-9fe3-28a4449ab6d9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.486427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.487841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.488295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.488766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.502015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:01.504556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.521592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20ef0d-b10d-4baa-baca-3aeaf62dfbc6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:01.531864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:01.532819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:01.533774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:01.540208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 332/649] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:01.540208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.589543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.590903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.591600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.591903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.611352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:01.614052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:01.644651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:01.645715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:01.646718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:01.646718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:01.646718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:01.646718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:01.651673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 318/650] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:01.657184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.660207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:01.704883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa07cbe-f722-44c2-b6a9-f2c95a58c115 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.714837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:01.771432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.772734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.773206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.773691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.775474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:01.776052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:01.778603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.800134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 333/651] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:01.802102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.814555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d39ef-8816-484a-89c6-04a9595e31ef None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:01.875309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:01.876484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:01.876951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:01.877437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:01.878811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:01.879352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:01.882001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:01.907520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 319/652] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:01.907520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:01.958330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d8e92-2e0b-4d95-86fe-85f249f84a12 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.006023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.006720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.007032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.007272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.008356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:02.008541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.010014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.019378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.026103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 334/653] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:02.027183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.058426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.069457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60ab0a-88a6-4bc6-965b-c2c9767d6e80 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.111784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.112460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.112746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.113024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.113779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:02.114083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.115513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.128388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 320/654] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:02.129242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.176492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b871bc-b03c-4816-9fde-016d271f10f2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.214308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.215124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.215405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.215710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.226795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.228428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:02.247923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:02.249753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 335/655] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:02.251554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.291905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dee3a-8eb6-4697-b365-45503bf33277 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.312773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.340624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.342008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.342506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.343002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.354775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.357540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.371170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e89a-16fa-4772-84d2-79bed9e850b2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-dd25c67f-5088-43b8-85e8-0b0a14b7deab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:02.385410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:02.387234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:02.388504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 321/656] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:02.390312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd801316-09c6-4ed4-b4ee-8592841fb32e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.390812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd801316-09c6-4ed4-b4ee-8592841fb32e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.391518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd801316-09c6-4ed4-b4ee-8592841fb32e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.393923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 322/657] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:02.395241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd801316-09c6-4ed4-b4ee-8592841fb32e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.419731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.424647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.425653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.426109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.426569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.427900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:02.428403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.430981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.445667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 336/658] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:02.445667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.545914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd801316-09c6-4ed4-b4ee-8592841fb32e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.604067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.605370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.605818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.606254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.618047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.621016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.627266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4be5a-40af-4846-9c20-c7182c7589fe None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.645481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:02.648638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:02.650148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:02.651642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 323/659] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:02.653105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.679504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.680256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.680580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.680793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.681480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:02.681790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.683197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba779d-7e6e-4e54-96a0-104f0a1f81f7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.701835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 337/660] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:02.704080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.704818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.705260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.733533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:02.735295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:02.789123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:02.843041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96586db9-4a67-47ba-873b-58f3d11114ae None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:02.860641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 338/661] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:02.862406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:02.897184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:02.898264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:02.898713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:02.899049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:02.899954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:02.900264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:02.902541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:02.937890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 324/662] 10.222.0.18 () {56 vars in 1139 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:02.937890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.028796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233173-3404-4504-83d7-d6c68ca671ac None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.033327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.083553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.084862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.085945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.086617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.088364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:03.089111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:03.089547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5dd76-44b4-4892-9b2c-c0e01e189113 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.092089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:03.118713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 339/663] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:03.120236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.138423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.150783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.152051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.152587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.153429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.165006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:03.167331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:03.186562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:03.196444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 325/664] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:03.198266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.280826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4a3d6-ab33-4f68-91ac-f73621ab0281 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.343306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.344663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.345127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.345573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.346045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d14299-c0a5-4240-b367-1296a4796f09 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.364361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:03.378171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:03.380363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 340/665] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:03.382034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.419409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.420245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.420720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.421189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.432300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.549724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.563696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93c792-4664-434d-8874-4e6b74e2fa19 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.569288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:03.572162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:03.608461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.609828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.609995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.610332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.611324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=f54544af7d84443e9b8fdbcb718bf824) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:03.611805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:03.613675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:03.627530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 341/666] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/role_assignments?user.id=f54544af7d84443e9b8fdbcb718bf824&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:03.627530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.775174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 326/667] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:03.776852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.829669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.866830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a643f-28f0-43f7-8ec4-32f44e905b7f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:03.939318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:03.940007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:03.940442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:03.940737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:03.942162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:03.942525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:03.944688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:03.958046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:03.968003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 342/668] 10.222.0.18 () {56 vars in 1137 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:03.969242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:03.977037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592b934-073d-4ee2-97f9-2858485d7cb8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.039214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.040421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.040898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.041326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.042140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:04.042891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.045288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.063603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 327/669] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:04.064684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.147467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51652162-1689-4e5b-9f87-589601706d32 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.207934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.209290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.209794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.210630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.232105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.234866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.235273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e3de0-126a-4e33-a382-09f103b651e5 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.249618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:04.267519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:04.269481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:04.271549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:04.271549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:04.271549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:04.271549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:04.274825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 343/670] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:04.276418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.317340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.319214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.320287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.321656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.378622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:04.444953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df367d1-53ab-480d-a19c-ff6756b93c21 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.450827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.452999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.502048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.503758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.503758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.503758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.518872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.521013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:04.546888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:04.547884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:04.548731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:04.548731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:04.548731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:04.548731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:04.553890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 344/671] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:04.554857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.588342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 328/672] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:04.589545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.647331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:04.652725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:04.744055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32646d-32e4-4ce4-a6e7-06633928c37e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.752553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae7e37-387f-4f4e-afa3-99c11b8cf28a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.792644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.793892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.794386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.794943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.796571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:04.797179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.799082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.818826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:04.819734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:04.820124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:04.820542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:04.826492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:04.826492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:04.826492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:04.827750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 345/673] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:04.829556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.842595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 329/674] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:04.849189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:04.971852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108133e-f2e4-487e-977f-9b2b091a64e8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:04.989222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:04.996256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31f5e4-fcf0-4ecf-aeb7-bd94f8e86527 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.008434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:05.017688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.018849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.019322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.019793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.028200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.029804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.036786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.037704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.037987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.038359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:05.046489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:05.047958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 330/675] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:05.049434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.055099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.056692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-3367cdd5-f389-4281-b526-c5f8f93d90c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:05.074713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:05.075987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:05.076846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:05.076846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:05.076846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:05.076846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:05.079677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 346/676] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:05.081769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea4d6-0157-48cd-b214-0e4a481ce706 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.082244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea4d6-0157-48cd-b214-0e4a481ce706 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.082614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea4d6-0157-48cd-b214-0e4a481ce706 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.084507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 347/677] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:05.085648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bea4d6-0157-48cd-b214-0e4a481ce706 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.207002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9c4f-abf2-492a-bd18-f1ec6354c593 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.222301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bea4d6-0157-48cd-b214-0e4a481ce706 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.266209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.266543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.267013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.267287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.267583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.267682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.268068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.268477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:05.268570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.268784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.270245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.281860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.284454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.287716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 348/678] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:05.289084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-342d1571-0611-44b3-a156-6d01e62abb8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:05.314407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:05.316256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:05.317848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:05.319526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 331/679] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:05.321765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.322262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.322755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.326071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:05.355139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:05.356957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:05.397326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:05.432548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d563989-e9a2-4e0d-b44c-14d0573573fa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.447543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 332/680] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:05.448023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.490705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.491778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.492277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.492730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.493954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:05.494433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.496777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.520266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 349/681] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:05.520266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.599800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b715b-3b48-42be-a3fa-a1cfeaba8ff7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.645036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:05.656218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.657160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.657581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.658023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.659135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=fcb115c5cad84f55b143c392c8c23dad) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:05.659629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.661257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.669479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 333/682] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/role_assignments?user.id=fcb115c5cad84f55b143c392c8c23dad&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:05.670369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.690417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2ffe7-c2e5-4ac6-9669-59be66f3b452 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.723849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:05.760625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.762480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.763548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.764463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.766084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:05.766670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:05.769320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:05.810164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 350/683] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:05.812372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.869321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe822735-d697-49fe-9703-2832e9762516 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:05.946116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:05.946931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:05.947209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:05.947511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:05.960846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:05.969021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:05.970037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 334/684] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:05.971240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:05.992348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d80a0e-840a-45e4-b123-a980bbbdf957 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.034553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.036163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.036635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.037056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.050963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.053887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.059082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5461c93a-2259-4879-be36-11c02f050a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:06.082243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:06.083330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:06.084247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:06.085751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 351/685] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:06.087612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7158c-1d4c-446b-b3da-786ef2b20881 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.088097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7158c-1d4c-446b-b3da-786ef2b20881 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.088604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7158c-1d4c-446b-b3da-786ef2b20881 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.097568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 352/686] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:06 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:06.099181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7158c-1d4c-446b-b3da-786ef2b20881 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.115055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.135380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66c2e-4b80-46a6-8778-d5dd1e6e9b95 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.184828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.185585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.186100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.186288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.281080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7158c-1d4c-446b-b3da-786ef2b20881 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.290309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.291837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990de3e9-547b-4ca7-9f3d-49847539704e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.329379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.330326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.330726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.331011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.332110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:06.332535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.334615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.347251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 353/687] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:06.348350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.425581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.434424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.447848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 335/688] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:06.449518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.450075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.450548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.469368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:06.470865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:06.508831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be268b-4bd3-476a-821e-0c2200b254c9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.555660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 336/689] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:06.555660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.579097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.580913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.581662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.582348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.584279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:06.585520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.588644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.611074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 354/690] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:06.612505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.732218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39463e-1c2d-47cf-9ca3-1722d7765bc0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.737304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f94dc-9fd5-4456-817d-948a5a906747 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.758525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.765424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:06.772261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.772955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.773238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.773515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.790200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.791797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.791958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:06.793204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:06.793557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:06.794275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:06.808006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:06.809419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:06.821125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:06.822286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:06.823361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:06.823361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:06.823361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:06.823361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:06.823361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 355/691] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:06.824667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:06.831873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:06.832885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:06.833891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:06.833891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:06.833891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:06.833891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:06.836170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 337/692] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:06.837250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:06.974625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aee2eb-f860-4a38-9454-41c26fea50b6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:06.989951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9fbc7-a6ae-4f98-ae20-95e11c24f711 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.028007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.029021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.029373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.029785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.033571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.034308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.034593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.035016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.036907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=5493d9cde9404225bedc423f0ecf213c) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:07.037521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.040776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.042921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.044640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.052119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 338/693] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/role_assignments?user.id=5493d9cde9404225bedc423f0ecf213c&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:07.053608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:07.094438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:07.095457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:07.096358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:07.096358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:07.096358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:07.096358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:07.097403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:07.098049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 356/694] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:07.099541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.164752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:07.209122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37049-2b85-4653-99e2-b66a69e9e1d6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.282295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.283116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.283276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.283780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.284806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:07.285348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.288117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.302825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 339/695] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:07.304089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.309206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d60887-898a-4510-992e-02bfb1960691 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.356260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.356852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.357127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.357554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.358133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:07.358440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.359974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.380950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 357/696] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:07.383447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.497241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:07.515298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb27e0-59e4-457a-9efa-16728b4540c5 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.530346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:07.557772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e985e6-88ab-4669-9442-e400a38e5a76 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.569256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.570853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.571827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.572546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.573841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:07.574387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.576661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.594593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 340/697] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:07.596475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.618897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.619464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.619777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.620043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.627812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.629291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:07.644193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:07.645292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:07.646368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:07.650416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 358/698] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:07.651382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.751316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0494-8cb6-4f10-b0f0-ee557ec09fbf None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.763529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746d4a-759b-48aa-b186-985398e9d201 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:07.792416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.793884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.794637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.795137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.801818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:07.802573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:07.802863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:07.803138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:07.803960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.805875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.806825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:07.808552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:07.827963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:07.836321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 341/699] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:07.838682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.849241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 359/700] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:02:07 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:07.850355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:07.854559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:07.862692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.014154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88731ad5-d5e8-44c7-9b7c-58e2afcf8b50 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.075555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8644-c3f7-4198-baee-c2c46b465eae None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.076795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.077591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.077900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.078135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.078940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:08.079247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.080771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eea621-5aeb-43fa-b29f-644854b3e188 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.098539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 342/701] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:08.100590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dcb9-26cb-477f-ba70-902b52a33bfa None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.100885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dcb9-26cb-477f-ba70-902b52a33bfa None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.101148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07dcb9-26cb-477f-ba70-902b52a33bfa None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.102862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 343/702] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:08.109047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07dcb9-26cb-477f-ba70-902b52a33bfa None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.144246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.144956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.145233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.145506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.146395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:08.146711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.148317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0049c4e-4e7c-49cf-978f-a5cf37d93d67 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.174881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 360/703] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:02:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:08.178615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.179334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.179980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.202051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:08.203407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:08.273625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.300088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.326024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 361/704] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:08.326024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.362305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07dcb9-26cb-477f-ba70-902b52a33bfa None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.423271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.425025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.425825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.426596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.428613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=d14dd264bd4343b49ef968789de08f73) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:08.429507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.432593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.445026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 344/705] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/role_assignments?user.id=d14dd264bd4343b49ef968789de08f73&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:08.447237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.461017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c28296-e0a8-4ae1-b9e5-0e70874e81ad None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.541594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.542201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.542484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.542751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.631521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541596e-dbdb-480d-9802-5905e306571a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.664244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.692893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.694555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.701935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.703779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.704678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.705341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] PATH_INFO: `/v3/services/network` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.707062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.710626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.718402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:08.732763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:08.733841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:08.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:08.742574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 345/706] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:08.744452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.860959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 362/707] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:08.866405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.899562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a6864-2896-46f7-b8da-ecb7de8cb972 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:08.939310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:08.940407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:08.940888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:08.941469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:08.942511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] RBAC: Adding query filter params (name=network) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:08.943142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:08.944829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:08.955395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 346/708] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:08.957369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:08.972137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:08.982832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d03c4b-43d6-4554-8635-b6430c00902f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.021886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:09.022758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.023046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.023321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.034088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:09.037821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:09.039319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:09.060096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:09.061062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:09.062431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:09.062431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:09.062431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:09.062431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:09.062431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 363/709] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:09.067306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:09.085402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7f76e-a21f-44e2-8d3c-7b42675fd704 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.127454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:09.128670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.129161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.129635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.130576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] RBAC: Adding query filter params (type=network) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:09.131148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:09.133011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:09.143286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 347/710] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:09.199357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c893e-48a3-43a3-88cb-84db5f2534e2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.241833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:09.242526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.242812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.243130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.243950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:09.244261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:09.245704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b82e1-a12f-4469-9df1-6aa3d4d167cf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:09.263891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 364/711] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:09.266074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:09.429287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a25911-8f1d-463b-ba8e-7568c9128952 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.471607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:09.483962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:09.485291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.485964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.486611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.497298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:09.500197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:09.503100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece818-6618-4ebe-837e-93352132b23b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.503388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece818-6618-4ebe-837e-93352132b23b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.503683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece818-6618-4ebe-837e-93352132b23b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.505081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 365/712] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:09.511953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.512624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.512969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.518381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:09.524578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:09.525613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:09.526920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 348/713] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:09.534855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:09.541892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:09.545812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:09.671985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 366/714] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 12:02:09.694267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:09.741626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a30ee-7f69-4ebf-8eab-68e128d88fb9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.794808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:09.795532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:09.795820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:09.796095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:09.796942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:09.797264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:09.798801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:09.826575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 349/715] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:09.836429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:09.905211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:09.933468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56fbed-8b6f-4171-8a3a-db5951d646d7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:09.992684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:10.010720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f30dc2-e799-4901-9c2d-905b5c02ddef None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:10.014900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:10.016350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:10.016865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:10.017342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:10.018624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:10.021284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:10.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:10.051743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:10.052603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:10.055160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 367/716] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Jul 21 12:02:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:10.062646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:10.081119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:10.081803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:10.082084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:10.082360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:10.083273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:10.084407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:10.086372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:10.094989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 350/717] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:09 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:10.203304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674e3f3-b96d-483d-8692-19c86a0b30a2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:10.243127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:10.243745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:10.244027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:10.244296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:10.245037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:10.245342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:10.246840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3a90-53e2-49dd-a1bb-770303a152e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:10.252800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 368/718] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:10.262406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:10.406288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5bef4-8b79-4baf-9e0f-7c0ee3eb8864 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:10.422121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:10.455753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:10.456453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:10.456737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:10.457014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:10.457750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:10.458057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:10.459468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1976a1-1637-4eaa-a5fc-c7d63d1457dd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:10.470373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 351/719] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:11.580560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61190566-4b1d-4db3-8772-347d809ea8f5 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:11.582126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61190566-4b1d-4db3-8772-347d809ea8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:11.583037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61190566-4b1d-4db3-8772-347d809ea8f5 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:11.588988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 369/720] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:11.597248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54ca8-f9f1-4d8c-9773-184970b17b2a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:11.598753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54ca8-f9f1-4d8c-9773-184970b17b2a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:11.599614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec54ca8-f9f1-4d8c-9773-184970b17b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:11.629125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-aec54ca8-f9f1-4d8c-9773-184970b17b2a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:11.631711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-aec54ca8-f9f1-4d8c-9773-184970b17b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:11.750767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 352/721] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:11.833259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61190566-4b1d-4db3-8772-347d809ea8f5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:11.905864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:11.972559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61190566-4b1d-4db3-8772-347d809ea8f5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:12.018610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:12.019435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.019755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.020010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.032082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:12.033743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111c00-5b3b-48b6-948a-b1db3f7164f8 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.034032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111c00-5b3b-48b6-948a-b1db3f7164f8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.034312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a111c00-5b3b-48b6-948a-b1db3f7164f8 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.036071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 353/722] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:12.040171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:12.042839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:12.044384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 370/723] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:12.045814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d778a-a058-4185-9fe9-585a643767ed None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.045814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d778a-a058-4185-9fe9-585a643767ed None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.051521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d778a-a058-4185-9fe9-585a643767ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.054381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:12.112166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-293d778a-a058-4185-9fe9-585a643767ed None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:12.114033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-293d778a-a058-4185-9fe9-585a643767ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:12.330660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156f2dc-214e-40b3-8f2b-3dd96c42472f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:12.346018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 354/724] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:12.370924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:12.407144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:12.408352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.408850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.409341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.410876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:12.411418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:12.414023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:12.440975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 371/725] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:12.442999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d778a-a058-4185-9fe9-585a643767ed None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:12.446269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:12.518237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:12.580267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3779dc3-9c5d-4b57-85da-9a0003503fda None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:12.638725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:12.642392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.644617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.645377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.672702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:12.675304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:12.678525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d778a-a058-4185-9fe9-585a643767ed None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-4a0b80dc-7545-4aff-9907-8d2f056a6dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:12.719696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:12.721668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:12.723671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:12.723671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:12.723671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:12.723671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:12.733509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 372/726] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:12.733509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd58e-ed12-43d5-9c6e-ca10a95a3cf0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.733509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd58e-ed12-43d5-9c6e-ca10a95a3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.733509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd58e-ed12-43d5-9c6e-ca10a95a3cf0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.737831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 373/727] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:12.742341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbd58e-ed12-43d5-9c6e-ca10a95a3cf0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:12.764713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:12.766678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.767174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.768045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.791961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-125541d6-5877-4ab2-a8f3-7800498c71cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:12.811307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:12.812378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:12.813197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:12.813827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:12.815225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 355/728] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:12.819766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:12.820258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:12.821171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:12.853651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:12.855923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:12.961586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbd58e-ed12-43d5-9c6e-ca10a95a3cf0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:12.998148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 356/729] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:13.000272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.042280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.043302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.043793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.044247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.045592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:13.046121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.050079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.059259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:13.087031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 374/730] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:13.088715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.222730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efeab9-0d5a-4032-bcd2-bad79084b439 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.267566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.268262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.269120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.269421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.270225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:13.270527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.272512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.287632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 357/731] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:13.287632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.312070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:13.327111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ca683-288f-402a-88ba-95a15bd7a37e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.386990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.387714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.388000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.388276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.389084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.390530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd1843-6b5e-4a42-a7c8-3a53453b0567 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.409697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 375/732] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:13 2022] POST /identity/v3/services => generated 240 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:13.411084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e8a43-ce35-48c6-965b-eac5cecab063 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.411452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e8a43-ce35-48c6-965b-eac5cecab063 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.411825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e8a43-ce35-48c6-965b-eac5cecab063 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.413477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 376/733] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:13.414269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e8a43-ce35-48c6-965b-eac5cecab063 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.466689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:13.496165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd25d-9d69-446d-a26c-0929e99f7696 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.556131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.556991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.557260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.557529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.567252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.569050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-91cf8e87-2332-4b9c-96af-bdb5a3444fd3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:13.586492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:13.588455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:13.590282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:13.593101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 358/734] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:13.594457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f600d-b3fc-4dbe-a935-2c75964f1c55 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.594747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f600d-b3fc-4dbe-a935-2c75964f1c55 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.595025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f600d-b3fc-4dbe-a935-2c75964f1c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.609141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e8a43-ce35-48c6-965b-eac5cecab063 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.615577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-767f600d-b3fc-4dbe-a935-2c75964f1c55 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:13.616220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-767f600d-b3fc-4dbe-a935-2c75964f1c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:13.670976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.671837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.672112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.672378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.686753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.688457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:13.708212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:13.709154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:13.710417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:13.710417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:13.710417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:13.710417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:13.710417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 377/735] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:13.711898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.722489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:13.739010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 359/736] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:13.739727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ddda-25ae-4c0b-a469-83df2259e177 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.740319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ddda-25ae-4c0b-a469-83df2259e177 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.740619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ddda-25ae-4c0b-a469-83df2259e177 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.742104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 360/737] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:13.742923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ddda-25ae-4c0b-a469-83df2259e177 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:13.843060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:13.875586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ddda-25ae-4c0b-a469-83df2259e177 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.896635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b333ea-90c1-4172-9db6-64a47ad35b1f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:13.917484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.918185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.918464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.918731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.919549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:13.919859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.921368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9375c0-ad7f-4e4d-8240-665df53e4f28 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:13.942191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 361/738] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:13.942191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.942191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.942191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.958816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:13.960814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:13.970663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:13.971908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:13.972444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:13.972885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:13.974537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:13.975084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:13.977957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:14.014010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 378/739] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:14.014010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.057211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 362/740] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 12:02:14.058573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.139804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:14.160159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:14.195106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3707-691b-4fc8-8e4a-f9f1c4171f68 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:14.218432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fac75-9dbb-4681-b942-10c3a93632de None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:14.254829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:14.255630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:14.255911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:14.256184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:14.272485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:14.272802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:14.273600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:14.273877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:14.274154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:14.283216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:14.284758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:14.285728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:14.287472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 379/741] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:14.290617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:14.315350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:14.317208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:14.320784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:14.326419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 363/742] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:14.326419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.437492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7bcbf-7b35-445a-8e59-e9a0dc52b0d4 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:14.484708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:14.488431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5534f-ecae-4496-bc47-8e333b5d1143 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:14.506459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:14.507617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:14.508124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:14.508806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:14.519311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:14.533436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:14.534121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:14.534410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:14.534658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:14.535438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:14.536818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:14.560246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 364/743] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:14 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:14.562927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.651828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:14.656550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:14.790737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dd0ba-5c65-4b34-963c-af1e4c65ac3b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:14.850296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 380/744] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 12:02:14.851710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.869326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:14.870247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:14.870697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:14.871141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:14.872387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:14.872880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:14.875248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:14.892165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 365/745] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:14.893798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:14.902345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:14.973038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.043031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2ff1-dc17-4507-b1a9-25eb035dc770 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.081095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.081928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a078-e24c-4743-9219-37171ab64279 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.082246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.082744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.083175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.084209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:15.084689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.086862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.107162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 381/746] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:15.108312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.147046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.148540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.149027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.149643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.168106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.171104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.200097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:15.205294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:15.206391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:15.207341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:15.207341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:15.207341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:15.207341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:15.209256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 366/747] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:15.210934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.222990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3488ff-1e21-47a6-919f-9ea339542dc4 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.262027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.263320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.263889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.264356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.276102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.277937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:15.298137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:15.299063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:15.299984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:15.299984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:15.299984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:15.299984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:15.302419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 382/748] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:15.303659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.388447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.401702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510d6d7-5d07-4a66-b551-481b9f799cb4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.416493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7f5f-48ce-4670-b087-cb6bbfe82455 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.455056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.455910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.456249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.456588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.457415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:15.457787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.459298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.461811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.474360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 383/749] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:15.475333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.481101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.482861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.483705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.484464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.621941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.624454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.633552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c281b1f-507b-4605-aada-3726b5ab219d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:15.698942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:15.701117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:15.702009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:15.703184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:15.704924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:15.705884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:15.709282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:15.741231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 384/750] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:15.743372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.800398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.827881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 367/751] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:15.829593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:15.869457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:15.975152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4ea67-7be9-4d1f-a86e-54fad0066309 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.023965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.024879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.025162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.025437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.033597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.035089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.048802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260f362-6074-416f-801e-42c52f11c254 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:16.051104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:16.052664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:16.054268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:16.057389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 385/752] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:16.058362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.122645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.124664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.125593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.126454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.142508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.163353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.174904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841dfe0-cc1c-4887-857e-ccc7974191ca None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.190451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:16.198479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:16.200281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:16.202028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:16.211939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 368/753] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:02:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:16.213477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.217595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.218477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.218763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.219042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.231954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.233477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:16.253352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:16.255106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:16.256876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:16.256876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:16.256876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:16.256876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:16.259678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 386/754] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:16.260528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.331076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:16.387933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4de57-46b2-478e-b325-b8e4d04ab22a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.391094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6f3f-d4d7-4dc9-92b8-7e73dd88e1ee None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.425360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.426061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.426345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.426620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.427510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:16.427826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.429534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.430366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.431434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.431970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.432535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.433544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:16.434046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.435740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.449679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 369/755] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:16.450988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.459032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 387/756] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:16.459032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.494375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:16.580972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05b2-4f8d-448a-afd1-797599cd0b11 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.598126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b71b9-6b22-4cc5-a4a0-939ea59d7e6e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.601084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:16.625759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.626643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.626754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.626991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.627778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:16.628117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.629540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.645327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.646111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.646390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.646663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.647544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:16.652536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.652735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 370/757] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:16.652735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.652878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640262df-886d-489d-982c-fbdb447bacc2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.657175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 388/758] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/role_assignments?user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:16.663537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5b71-106c-4c7b-9097-0217e0aa0400 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.663537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5b71-106c-4c7b-9097-0217e0aa0400 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.663537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c5b71-106c-4c7b-9097-0217e0aa0400 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.663537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 389/759] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:16.663537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c5b71-106c-4c7b-9097-0217e0aa0400 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.804858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:16.817481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754047dc-567a-40b8-9448-251cbfc0708f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.830955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c5b71-106c-4c7b-9097-0217e0aa0400 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:16.859675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.860516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.860946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.861407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.874571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.876634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.897955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-25e39a0e-5634-46e5-91b7-041a581c3b7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:16.899449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:16.901160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:16.904270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:16.904270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:16.904270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:16.904270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:16.904940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.904940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.904940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.909961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 371/760] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:16.909961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:16.909961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:16.909961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:16.916030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:16.918686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:16.929260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:16.930968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:16.945540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:16.948232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:16.950992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 390/761] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:16.960361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:16.960567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.051670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 372/762] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:17.053605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.093352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07439802-f243-4225-9c8d-ce68b153d51d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.151683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.152483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.152829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.153193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.154038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:17.154508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.162287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947438f0-3cf1-4372-a791-fc084b47a6fa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 391/763] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2578-cebd-4fdb-ac58-2e60bb92a139 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2578-cebd-4fdb-ac58-2e60bb92a139 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd2578-cebd-4fdb-ac58-2e60bb92a139 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 392/764] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:17.183531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd2578-cebd-4fdb-ac58-2e60bb92a139 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.197469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.240476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507c12-62ab-489a-ae27-aa26c09355b9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.299215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.299935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.300270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.300538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.304733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:17.304733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.304733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.317378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 373/765] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:17.318283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.345236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.348928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd2578-cebd-4fdb-ac58-2e60bb92a139 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.389293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.393323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.393787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.394165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] PATH_INFO: `/v3/services/network` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.394857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.396337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-84dab109-c9ba-453a-9ce3-eb01be100706 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:17.417533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:17.418782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:17.419656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:17.420384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 393/766] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:17.422485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.422985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.423472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.450890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:17.451305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e5154-be84-4517-8b52-4e21cdf2ad0e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.452516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:17.490568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.491387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.491708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.491973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.499702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.501202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:17.516313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:17.517896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 374/767] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:17.519629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.567854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 394/768] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:17.569000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.571649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.624762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.655986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3e129-a07d-46ed-b352-e622dfc0ace1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.697580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.698635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.698987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.699262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.700203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=d14dd264bd4343b49ef968789de08f73) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:17.700520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.701975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.719528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 375/769] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/role_assignments?user.id=d14dd264bd4343b49ef968789de08f73&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:17.719528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.734811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcf5ce-3134-4945-adac-c2d64183ac0b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.786688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.787535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.787887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.788226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.789076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] RBAC: Adding query filter params (name=network) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:17.789443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.791187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.797459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 395/770] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:17.798632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.871689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e245efe-3288-440e-983e-a73e9929d219 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.909637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.932376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.933220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.933637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.934021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.934935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:17.935653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:17.937396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:17.956917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 376/771] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:17.956917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:17.956917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:17.958649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878c21a-0d07-47a5-936b-9205e0229350 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:17.996926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:17.998100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:17.998563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:17.999014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:17.999914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.001695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:18.015702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:18.016872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:18.017815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:18.020976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 396/772] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:18.022263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:18.114360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b873ff-2a29-499b-b323-99f249b9d138 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.140444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506e24-b3bf-4570-84b1-ebd42211e5c9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.170209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.170833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.171119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.171400 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.172492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] RBAC: Adding query filter params (type=network) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:18.172824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.174691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.182305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 377/773] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:18.183519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:18.193031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:18.196930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.198699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.199360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.199987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.204341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=bccc158b75eb41ca8431bbb67af65260, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.205848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.222854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 397/774] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:02:18 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:02:18.229067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:18.332453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:18.430025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5ab17-3445-40cb-969e-c3588ba84aae None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.458251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d6a4d-3fc0-4840-9d9e-b3aba262eab5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.502255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.502943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.503222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.503552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.504261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:18.504561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.505425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.505975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.506136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.506381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.506643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.507468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] RBAC: Adding query filter params (interface=public, service_id=767f1a41637c4d1ea111fe1bf5821896, region_id=RegionOne) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:18.507815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.509328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.513384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 378/775] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:18.519987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 398/776] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:18 2022] GET /identity/v3/endpoints?service_id=767f1a41637c4d1ea111fe1bf5821896&interface=public®ion_id=RegionOne => generated 194 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:18.520638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:18.524992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:18.614795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:18.699366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39def40c-35a3-449c-b8c2-8cf39a098206 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.749490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b8c3-f1e8-4138-b1a4-ccab8cf67cd6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:18.775430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.776261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.776548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.776821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.777539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.778990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.796085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 399/777] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:18 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:18.800038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:18.815190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:18.816487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:18.817060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:18.817621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:18.818942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:18.819666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:18.822512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:18.839888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 379/778] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:18.845867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:19.022012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89eab69-2406-4ca6-a753-4fbc13dde54b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:19.064463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:19.107929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:19.109012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.109465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.109937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.111462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] RBAC: Adding query filter params (interface=public, service_id=e66b7bc398a041afa82f38c0d5b1196d, region_id=RegionOne) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:19.112013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:19.114655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2d13a-6587-4744-83a7-cd65747c7ae7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:19.125702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 400/779] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:18 2022] GET /identity/v3/endpoints?service_id=e66b7bc398a041afa82f38c0d5b1196d&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:19.131513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:19.209082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a973-2626-4093-8d9b-532c4c06b099 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.209574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a973-2626-4093-8d9b-532c4c06b099 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.210418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a973-2626-4093-8d9b-532c4c06b099 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.214844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 401/780] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:19.225103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.225634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.226130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.255306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:19.257388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:19.276923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc58c-2e82-4c2c-adba-1889a1838df8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:19.314908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:19.315645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.316032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.316319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.317097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:19.318632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:19.325122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 380/781] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:19 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:19.412060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 402/782] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:19.476210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:19.560451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:19.613342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938020-f234-46e1-9622-7a60fbc7511d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:19.654479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:19.655310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.655650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.655903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.668915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:19.677034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:19.678920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:19.680693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 381/783] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:19.687549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:19.797766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a58896-b4ce-4efa-a0e8-31666b511a61 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:19.816048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:19.859342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:19.860075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:19.860349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:19.860605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:19.861380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:19.861681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:19.863083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:19.873060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 403/784] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:19.878411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:20.055683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41fa4-d415-4e0f-b85b-313c13890cbe None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:20.078593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:20.104775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:20.105624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:20.105902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:20.106173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:20.128343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:20.129852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:20.164381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:20.166831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:20.168535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:20.168535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:20.168535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:20.168535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:20.171635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 382/785] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:20.176730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:20.378851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5d89-ebf0-49fd-b255-76363e6ea2af None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:20.462637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:20.463346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:20.463667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:20.463936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:20.464697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:20.465006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:20.466433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:20.489672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 404/786] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:20.495600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:20.659944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f5d8b-be86-4fab-88b8-52589a0fc1cb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:20.698012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:20.699052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:20.699811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:20.700128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:20.709504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:20.711350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-e8bc8b3e-449b-44f7-a7ab-dad7268baa83 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:20.727928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:20.729745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:20.731375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 383/787] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:20.736288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:20.837647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:20.961532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af167c-b21e-424a-9660-29d71d6480ce None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:20.966339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4c2f2-5ef6-4b10-9fdc-1b27632c0fdd None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:20.966791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4c2f2-5ef6-4b10-9fdc-1b27632c0fdd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:20.967251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4c2f2-5ef6-4b10-9fdc-1b27632c0fdd None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:20.970148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 384/788] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:20.980292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:20.981453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:20.981936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.003151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.003868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.004146 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.004416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.005273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:21.005568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:21.007049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:21.027744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 405/789] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:21.028694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:21.030305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:21.047888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.068023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:21.157359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:21.157917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 385/790] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:21.168301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7a065-f2a3-422f-aef3-bca00ee88dae None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:21.207286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314597c-9719-4888-a048-61335e720f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.208876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314597c-9719-4888-a048-61335e720f05 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.209350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314597c-9719-4888-a048-61335e720f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.210150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314597c-9719-4888-a048-61335e720f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.222188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314597c-9719-4888-a048-61335e720f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:21.224000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314597c-9719-4888-a048-61335e720f05 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:21.235695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f314597c-9719-4888-a048-61335e720f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:21.244103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:21.245047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:21.245902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:21.245902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:21.245902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:21.245902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:21.248171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 406/791] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:21.254534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314597c-9719-4888-a048-61335e720f05 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.399437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314597c-9719-4888-a048-61335e720f05 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:21.437405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.438083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.438364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.438636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.439378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:21.439712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:21.441143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:21.449623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232745a-8f92-4b7a-81f2-87c60e81ae71 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:21.454610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 407/792] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:21.460887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.469992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:21.525901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.526650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.527003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.527284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.555404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:21.570897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:21.572016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:21.572842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:21.575541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 386/793] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:21.585972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.661314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:21.694764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fb06-c0db-4e0e-bfbe-be486f732f4b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:21.798984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.800218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.800701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.801193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.816264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:21.818133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:21.847344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277909ab-4a1a-4486-a2b5-b7f9a20bd11c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f6fead67-5f57-4308-b4c7-dcc9cc3d316a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:21.871811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:21.872750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:21.873616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:21.874583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 408/794] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:21.881369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac39bd-509e-4c43-94bf-b9854b281ef0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.882090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac39bd-509e-4c43-94bf-b9854b281ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.882581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac39bd-509e-4c43-94bf-b9854b281ef0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.884746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 409/795] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:21.885776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac39bd-509e-4c43-94bf-b9854b281ef0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:21.915683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:21.916846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.917325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.917815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.919168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:21.919757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:21.922676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6cef9-9641-481b-a386-15f2943831ec None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:21.940168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 387/796] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:21.942508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:21.942959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:21.943405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:21.981239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:21.983133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:22.011937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.079311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac39bd-509e-4c43-94bf-b9854b281ef0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.083591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 388/797] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:22.085264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.112245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.162284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.163437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.164637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.164637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.166036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:22.166522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:22.169553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:22.192452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 410/798] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:22.193840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.248017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0d681-0a4c-4e39-8e9a-929a4958918c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.310960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.312381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.312843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.313335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.332525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:22.335590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:22.337125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3ae7-978f-45d3-b25f-40cce2f4eb90 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.365726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-4a6e3a5f-b363-4c18-9d1d-e13a51aad7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:22.368898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:22.370622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:22.372911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:22.372911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:22.372911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:22.372911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:22.372911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 389/799] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:22.373546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56c4f-b16e-4925-9f90-63051723d8e4 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.374069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56c4f-b16e-4925-9f90-63051723d8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.374450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56c4f-b16e-4925-9f90-63051723d8e4 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.377510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 390/800] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:22.378971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca56c4f-b16e-4925-9f90-63051723d8e4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.390237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.391055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.391334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.391655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.404349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:22.412135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:22.413982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 411/801] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:22.415752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.507718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.557170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca56c4f-b16e-4925-9f90-63051723d8e4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.591222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02540361-e9b9-4d26-9c20-f1c4d5d69f73 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.602664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.603850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.604246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.604648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/7da32d4f2caf4f0aa22f839c4a3e6b6b/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.624741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=7da32d4f2caf4f0aa22f839c4a3e6b6b, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:22.626583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d3e42-4a69-454d-ae4d-d60b72d85b14 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:22.637036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.638077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.638494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.638945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.640187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:22.640695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:22.643029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:22.647403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 391/802] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:02:22 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/7da32d4f2caf4f0aa22f839c4a3e6b6b/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:02:22.659742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca2de-f896-45a3-bb4b-433104ab0a90 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.659742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca2de-f896-45a3-bb4b-433104ab0a90 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.659742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca2de-f896-45a3-bb4b-433104ab0a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.662836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 412/803] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:22.662836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.681238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-88bca2de-f896-45a3-bb4b-433104ab0a90 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:22.682726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-88bca2de-f896-45a3-bb4b-433104ab0a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:22.746533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.802977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 392/804] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:22.805348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afe5d5-81d7-4538-9397-0e7f8f3e3e94 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.806078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afe5d5-81d7-4538-9397-0e7f8f3e3e94 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.806549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afe5d5-81d7-4538-9397-0e7f8f3e3e94 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.808255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 393/805] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:22.809868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afe5d5-81d7-4538-9397-0e7f8f3e3e94 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:22.853050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1934154-47a1-4858-be0a-03df18bf7e79 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:22.889701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:22.920371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:22.921236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.921567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.921895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.922848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:22.923242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:22.924827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4605d9-2986-4628-88b0-a3e788b9a02e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:22.941064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 413/806] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:22.943940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:22.944452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:22.944946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:22.974366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:22.976446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:22.982356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afe5d5-81d7-4538-9397-0e7f8f3e3e94 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.029507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.030344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.030623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.030896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.040380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.042225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:23.059141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:23.061036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:23.062826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:23.065652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 394/807] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:23.066669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.102726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 414/808] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:23.104222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.167912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:23.227149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f6aa4-3d6a-436f-b121-c23c7b87a0e7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.229527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:23.282720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.284304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.284643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.285041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] PATH_INFO: `/v3/services/network` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.285943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.287791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.299646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e32392-af1a-4022-bf1c-a150fa5ee4ad None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:23.303457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:23.304567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 12:02:23.305466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:23.306269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 395/809] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:23.314004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.366361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.367707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.368305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.368817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.388628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.391300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:23.428488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:23.430445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:23.432967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:23.432967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:23.432967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:23.432967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:23.438781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 415/810] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:23.440284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.450717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951d28f-a84a-4503-961e-3e9f643b8d05 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.499443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.500523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.500918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.501310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.502606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:23.503046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.504935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.535547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 396/811] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:23.535547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.571384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:23.620019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:23.634652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5795-9fb2-47bb-b02e-caffc5ef93d5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.676673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.677591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.677877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.678152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.678834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.680329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:23.694393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:23.696460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:23.698229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 416/812] 10.222.0.18 () {56 vars in 1129 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:23.699827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.719764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1627-b92a-4aa9-9d75-8d815aa6afa1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.767287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.768779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.769482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.769962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.771244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] RBAC: Adding query filter params (name=network) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:23.771801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.774482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.783675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 397/813] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:23.792478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.882143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4ad4-8497-4af4-8850-21d401b7c730 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.928042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:23.928727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:23.929064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:23.929367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:23.930164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:23.930521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:23.931834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:23.933593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:23.958808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 417/814] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:23.961032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a734707-939e-4107-b393-67ab7f8227bd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:23.962659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:23.991973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.011914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.013296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.013751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.014204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.027066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.028908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:24.057634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:24.059407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:24.061095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:24.061095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:24.061095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:24.061095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:24.061095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 398/815] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:24.061637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.119946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a0490-48fb-4014-ae6a-65d86ffb6161 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.171650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.172771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.173242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.173698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.174909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:24.175627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.178072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.186982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 418/816] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:24.193763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.223594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f06bd-fdb8-4156-bd65-e09782201ebd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.271668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.272933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.273208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.273478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.274288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:24.274637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.276027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.289318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 399/817] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:24.289318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.294356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.325163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.362409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cd07-b0a5-43a5-8d34-e0a051f111fb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.406471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.408981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.409478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.409942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.424579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.427121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.446428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ec229-f6f3-4254-bb30-1e075cb496c9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:24.453989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:24.455914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 419/818] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:24.458288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.496301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.497108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.497469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.497808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.498635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:24.499000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.500536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.515577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 400/819] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:24.516195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.593046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.643052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b62f-9f43-407f-8d8d-9670ee260cb2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.650249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c2f0d-5de7-4927-a9d3-e55f6154542c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.700784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.702011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.702495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.702980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.704418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.707359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.715010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.716173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.716612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.717034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:24.718214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:24.718691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:24.721195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:24.727070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.735058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 420/820] 10.222.0.18 () {56 vars in 1124 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:24.737047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.743326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 401/821] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:24.744243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:24.917392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c2072-80f9-430e-a4a9-983522bc6ec8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.941180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:24.967874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc783828-20a0-480d-8e71-0cd9610c53e0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:24.988141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:24.991185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:24.992044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:24.992884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.009030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.010890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.011626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.011909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.012184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.013044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:25.013351 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.014152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.015804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.037325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 421/822] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:25.037932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:25.046092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:25.047849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:25.049570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:25.050395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 402/823] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:25.053606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.131981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:25.198999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4be93e-2380-458f-9225-4e0ffad00577 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:25.246117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.246678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.246961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.247237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.248041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.249469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.259724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0faf-8bcd-4445-8d81-8667a59aa655 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:25.272333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 422/824] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:25.273309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.332951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.334072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.334567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.335409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.404888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:25.431713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813d161-b7be-450e-806f-f922b48f38aa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:25.441919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.443541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b201556-5dd2-4a62-8a9a-130c953f6c60 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.461077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:25.497310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.498357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.498863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.499349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.500814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:25.501288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.503950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023694f9-2002-43b5-abff-b912a33cbf9d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.526232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 423/825] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:25.528533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819e29e-1552-4461-b34f-1f08c6d9dc78 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.528991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819e29e-1552-4461-b34f-1f08c6d9dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.529475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819e29e-1552-4461-b34f-1f08c6d9dc78 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.532016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 424/826] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:25.534731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819e29e-1552-4461-b34f-1f08c6d9dc78 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.589037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 403/827] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:25.590399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.590688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.590958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.613097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:25.614365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:25.692052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 404/828] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:25.697152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.697856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:25.707536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819e29e-1552-4461-b34f-1f08c6d9dc78 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:25.741875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f393b-bddc-4611-87af-758098adc660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.742856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f393b-bddc-4611-87af-758098adc660 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.743137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f393b-bddc-4611-87af-758098adc660 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.743420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f393b-bddc-4611-87af-758098adc660 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/d14dd264bd4343b49ef968789de08f73/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.745526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f393b-bddc-4611-87af-758098adc660 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=d14dd264bd4343b49ef968789de08f73, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.746989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f393b-bddc-4611-87af-758098adc660 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.771782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 425/829] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:02:25 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/d14dd264bd4343b49ef968789de08f73/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:02:25.771782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f393b-bddc-4611-87af-758098adc660 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.846707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:25.862301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fd15e-a002-4c45-ba11-36c998baeeb6 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:25.902508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:25.903361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:25.903678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:25.903956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:25.916824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:25.918397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:25.939119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:25.940380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:25.941571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:25.941571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:25.941571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:25.941571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:25.942145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 405/830] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:25.947845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:25.982216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f393b-bddc-4611-87af-758098adc660 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.032642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:26.048267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.049733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.050554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.051084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.076721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:26.092532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d272c-3250-4e9a-961b-9d6b097e43b8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:26.096121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:26.097182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:02:26.098022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:26.100072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 426/831] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:02:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:26.111617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.152420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.153831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.154348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.154860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.155959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:26.159538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.159538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.177876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 406/832] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:26.187135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.344976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cd9b0-4866-4e03-9b22-0ad4a2d7e10b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.345824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:26.354218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f0bfc-68c6-4bb3-a4b3-9d478d674caa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.378020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:26.397318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.398659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.399228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.399978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.401564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:26.402367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.405302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.405583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.406481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.406903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.407313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.415688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.418245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.426496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 427/833] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:26.428209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:26.444105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:26.445100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:26.446037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:26.451834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 407/834] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:26.455112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.563791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8af4c7-6107-47dc-93bb-e8dbe3b084cf None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.607794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0d6f7-bc8f-48ec-896a-ccf1c470dcd7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.617761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.618910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.619953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.620412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.636867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.639562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.651956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.653082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.653657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.654081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.655711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:26.656334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.659158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:26.661534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:26.662390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:26.663289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:26.663289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:26.663289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:26.663289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:26.670669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 428/835] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:26.671212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.683230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 408/836] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:26.693434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:26.697651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:26.737301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:26.827243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7b1c-80d8-4b23-b710-437b60af480a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.839835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63419038-3643-4dc0-bcf9-787898375d00 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:26.875889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.876764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.876968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.877335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.878278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:26.878697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.880380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.883713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:26.884807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:26.885269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:26.885746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:26.886733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:26.887180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:26.888758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:26.896038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 409/837] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:26.907457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 429/838] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:26.913789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.071861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:27.077759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bca1d-d839-4b7a-bb32-817d33997e32 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:27.122969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:27.123880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.124150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.124428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.133880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:27.135454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:27.152326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:27.156868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:27.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 410/839] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:27.168619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.294793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:27.336872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95b9aa-b444-4d24-bbc7-583b54d38327 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:27.377395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:27.378502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.379003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.379545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.380619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:27.381148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:27.382951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54320e-2d04-48de-8be9-f18ddbfa862e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:27.404215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 430/840] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:27.413490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.541059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fb4d4-d67a-465b-a29b-0126053edd1d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.541322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fb4d4-d67a-465b-a29b-0126053edd1d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.541603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fb4d4-d67a-465b-a29b-0126053edd1d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.543023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 431/841] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:27.549033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.549322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.549768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.560230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e48ebc-ef10-4252-a28a-aec94ea293e9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:27.568159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:27.569523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:27.607615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:27.608407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.608692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.608981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.655711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 432/842] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:27.673186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.673838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:27.746960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:27.749893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:27.751971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:27.830743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284635d-801f-4754-9359-6a6c7e06cc03 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:27.884501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 411/843] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 12:02:27.887586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:27.888904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.889344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.889764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.891058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:27.892596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.893808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ec24e17e-e85f-4c33-8a49-f241e875d00b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:27.917225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:27.918306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:27.919142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:27.919927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 433/844] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:27.921476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e57ee-9a47-4d3f-a1cd-c5cf7638b268 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:27.921932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e57ee-9a47-4d3f-a1cd-c5cf7638b268 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:27.922374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e57ee-9a47-4d3f-a1cd-c5cf7638b268 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:27.924769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 434/845] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:27.926222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e57ee-9a47-4d3f-a1cd-c5cf7638b268 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:27.970129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:28.023126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52765-7b00-495a-8796-c7ea3bf6c06d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.089940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.091167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.091623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.092034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.112672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:28.115342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-fc7b507f-4a66-465b-b739-a0d5184f6fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:28.150469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:28.152197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:28.153948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:28.153948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:28.153948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:28.153948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:28.153948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 412/846] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:28.155284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.155284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.157352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.184487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:28.186456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:28.205328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e57ee-9a47-4d3f-a1cd-c5cf7638b268 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.233579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:28.257304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.258060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.258288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.258671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.259384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:28.259728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:28.261133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:28.267449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 435/847] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:28.268827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:28.301719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 413/848] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:28.302657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:28.376251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:28.412191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6f27f-2b83-4909-bb54-68a6479b4cd2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.454624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b114-d313-4360-be57-e10d97de451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.455202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b114-d313-4360-be57-e10d97de451b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.455470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b114-d313-4360-be57-e10d97de451b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.455858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574b114-d313-4360-be57-e10d97de451b None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.456611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b114-d313-4360-be57-e10d97de451b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:28.456924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b114-d313-4360-be57-e10d97de451b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:28.458343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f574b114-d313-4360-be57-e10d97de451b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:28.462240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbbb36-31b0-4f1f-8cdb-e2f8d049ea99 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.477625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 436/849] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:28.477625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b114-d313-4360-be57-e10d97de451b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:28.523040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.524093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.524545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.524989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.526213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:28.526723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:28.528879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:28.547544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 414/850] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:28.547544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:28.559451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:28.670544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574b114-d313-4360-be57-e10d97de451b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.716177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bda07a-ae1f-4ec7-8c0d-c0a871e352ec None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:28.720807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:28.754034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.755368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.755852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.756295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.777610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:28.792100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:28.793103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:02:28.793911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:28.794621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 437/851] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:28.805515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:28.891213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:28.892364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:28.892796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:28.893214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:28.902412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:28.904372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:28.921369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:28.922331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:28.923194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:28.930568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 415/852] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:28.932065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:29.022596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d131-2b41-48fc-97e6-8f9c7699d093 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.027362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:29.082229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.083371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.083904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.084391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.085857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:29.086399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.088236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:29.099354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 438/853] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:29.110343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:29.113649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631761f6-29b0-40da-a10f-7026b5253671 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.181612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.182466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.182868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.183248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.184222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:29.184638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.186223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:29.205755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 416/854] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:29.207053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:29.239534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:29.319219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108ecff-067d-442a-82a5-9b590954759a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.366084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4d9d-0348-4f9e-82ac-eba41bfdd394 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.380347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.381902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.382333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.382734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.404388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.406879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:29.434552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.436169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.436862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.437564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:29.438055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:29.439697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:29.441369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:29.441369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:29.441369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:29.441369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:29.444120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:29.445053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.447821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 439/855] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:29.448733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:29.450434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:29.462568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 417/856] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:29 2022] GET /identity/v3/role_assignments?user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:29.488898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:29.653349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5076b9-b0e7-4727-a59a-32fcae9a0b48 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.716411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.717498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.717988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.718473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.719860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:29.720371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.722841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142dfd26-0c64-40bb-bbcc-e9b3e6ec1d47 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:29.748610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 440/857] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:29.756089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:29.873391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:29.898688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409fca3-0e54-4825-9ef9-c3ee77eb35f8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:29.967923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:29.969258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:29.969767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:29.970276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:29.986054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:29.988603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:30.002207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadce289-f144-44c1-875e-4627d807ec59 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.002722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadce289-f144-44c1-875e-4627d807ec59 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.003212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadce289-f144-44c1-875e-4627d807ec59 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.006598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 441/858] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:30.015746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.016670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.017391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:30.028216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:30.030117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 12:02:30.031984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:30.035692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 418/859] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:02:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:30.040815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:30.054851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:30.057688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:30.194476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb24b57-d680-4ff5-9be8-243dda1efbb0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:30.245920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 442/860] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:30.250323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:30.251878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.252547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.253184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.254812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:30.255588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:30.258468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:30.299683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 419/861] 10.222.0.18 () {56 vars in 1135 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:30.302736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:30.314542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:30.328337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:30.349353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:30.574382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd487-329c-4fae-8348-dccfcb0a03f8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:30.629180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0118979-c350-4ab7-a2a3-9a446f5d542f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:30.639452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:30.641116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.641875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.642642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.734926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:30.737497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.738300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.739078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.744386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:30.747360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:30.772132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:30.774744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:30.777706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:30.780674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:30.781628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:30.784224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 443/862] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:30.798852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:30.799131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:30.875324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:30.909999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87209d4-66be-49da-96e0-71cd4f8ea635 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:30.948607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:30.949296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:30.949605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:30.949904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:30.950601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:30.950914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:30.952368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:30.963516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 444/863] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:30.969117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:30.990355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 420/864] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:30.999405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:31.092361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1242ec-ebd9-4322-bbd3-89d31ee27e1d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:31.137335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:31.138187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.138671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.139054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.140567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:31.144865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:31.146462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:31.152714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:31.175520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 445/865] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:31.211320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b6b2f-1385-4384-a88f-3d6f7c621bc4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:31.221969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:31.268374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:31.270390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.271296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.272080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.293909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:31.297391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:31.334104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:31.336310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:31.338170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:31.338170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:31.338170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:31.338170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:31.344549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 421/866] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:31.354406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:31.535386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405af2-ffde-46c9-a287-0b12acc8bd18 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:31.596096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:31.597179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.597642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.598101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.599443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:31.599991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:31.602520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbd00-79a9-45cd-97d6-fd8f870c1cb5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:31.631649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 446/867] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:31.637446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:31.716959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce375b2-9298-4966-8c1b-d8ae3f5d0e39 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.717977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce375b2-9298-4966-8c1b-d8ae3f5d0e39 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.718853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce375b2-9298-4966-8c1b-d8ae3f5d0e39 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.722417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 447/868] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:31.729605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.729910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.730222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.756320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:31.759733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:31.803692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db116-bded-47e2-ac78-e8605d0f794b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:31.847810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:31.849086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.849362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.849639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.859180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:31.860799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:31.873077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-9bdd16bc-826c-47ae-884a-cd374ea22509 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:31.878579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:31.885452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 422/869] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2425-5124-465e-a3eb-54dcfcf389e1 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2425-5124-465e-a3eb-54dcfcf389e1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.890033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2425-5124-465e-a3eb-54dcfcf389e1 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.890033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 423/870] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:31.894480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:31.894480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:31.894480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:31.898704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 448/871] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:31.901367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:31.921988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:31.924710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:31.953154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:32.025807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 424/872] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:32.027778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:32.079436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079bb4c-46c0-4b78-b533-03341c1a2898 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.142835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.143805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.144228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.144596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.145801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:32.146282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:32.149806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:32.173058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 449/873] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:32.173058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:32.204064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb1f30-4d66-4b56-a53c-b45d9e741b76 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.252774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.253661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.253932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.254243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.261469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:32.267695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:32.277608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:32.279450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 425/874] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:02:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:32.281110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:32.294756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:32.357879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8250728-a7fa-45a6-8401-6feb5405d10b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.417852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.418554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.418847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.419151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.419888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:32.421313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:32.451344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695504d-0937-44e5-99fd-d9b286d3f838 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.455303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 450/875] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:32 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:32.456880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:32.546983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.548230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.548759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.549202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.551004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=d14dd264bd4343b49ef968789de08f73) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:32.551584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:32.554452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:32.568330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 426/876] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:32 2022] GET /identity/v3/role_assignments?user.id=d14dd264bd4343b49ef968789de08f73&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:32.697908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf707-d6ef-459a-ac21-5597725bb822 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.721168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:32.769389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.770570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.771045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.771656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.773092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:32.773659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:32.776368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:32.793182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 451/877] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:02:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:32.799677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:32.881932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:32.922318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09599d97-7f8f-4785-8e56-ea2eb93b434e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:32.960892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:32.961750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:32.962027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:32.962297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:32.974454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:32.976052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:32.995354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:32.997166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:32.998924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:32.998924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:32.998924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:32.998924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:32.998924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 427/878] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:33.002786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:33.129522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b2791-c614-4494-b705-29113f8106a0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:33.179009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:33.179735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:33.180021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:33.180388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:33.181142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:33.181449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:33.182877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:33.198306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 452/879] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:33.204054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:33.322726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a182b-96ee-4541-923a-81d3f41efee2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:33.345717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:33.362469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:33.363313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:33.363672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:33.363948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:33.372949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:33.374538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:33.391373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:33.392394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:33.393303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 428/880] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:33.400691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:33.443435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:33.527391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7e92f-87a8-407c-9738-0a2169f8577b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:33.566461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:33.567163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:33.567447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:33.567756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:33.568618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:33.568931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:33.570399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:33.591456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 453/881] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:33.600640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:33.712123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98621cb3-88fd-4a31-9502-dbb078f507bc None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:33.750650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:33.751322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:33.751638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:33.751915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:33.807069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:33.828736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:33.830254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62cb33-9611-48a8-9fe7-0824f22751d2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:33.944881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 429/882] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:33 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:33.950905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:34.004981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba691a-bc31-45d9-a237-1c138946d745 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.005478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba691a-bc31-45d9-a237-1c138946d745 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.005966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba691a-bc31-45d9-a237-1c138946d745 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.009142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 430/883] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:34.017063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.017357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.017595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.038752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:34.040073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:34.080472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:34.102106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09282e-3fd4-4556-9b98-a8eea4424242 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:34.169052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 431/884] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:34.183387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:34.185337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.186080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.186800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.203708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:34.204134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:34.206257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:34.228615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:34.236037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:34.237018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:34.237904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:34.237904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:34.237904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:34.237904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:34.241098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 454/885] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:34.247746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:34.361312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3bbd8-3a43-4935-930d-cf9ae65468bf None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:34.412771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:34.413600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.413955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac90c0-7a1e-48c1-a3c0-b5c1168cad3f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:34.414112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.414266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.415161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:34.418085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:34.441462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 432/886] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:34 2022] POST /identity/v3/services => generated 269 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:34.477299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:34.478312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.478734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.479255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.480582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:34.481047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:34.483449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:34.496188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:34.506039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 455/887] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:34.514757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:34.606628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:34.651029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97894828-25ea-4e97-ae16-110c6105734c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:34.691787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:34.692651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.692932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.693209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.702801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:34.704480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:34.721300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:34.722258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:34.723131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 433/888] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:34.729239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:34.853006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f078-ba07-48f7-b22d-6191812d6c62 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:34.893254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:34.893948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:34.894300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:34.894583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:34.895428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:34.895781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:34.897279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7ece-be2a-4bcd-b7b5-793d58d699e1 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:34.913811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 456/889] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:34.920176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:35.040106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263adff-436d-4020-8fea-2f6ab9257836 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:35.073495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:35.077416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b713bb-bc87-4ec0-8957-ea267b9808a4 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.079630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b713bb-bc87-4ec0-8957-ea267b9808a4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.079630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b713bb-bc87-4ec0-8957-ea267b9808a4 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.079783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:35.080338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.080623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.080903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.081316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 457/890] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:35.081803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] RBAC: Adding query filter params (role.id=ec07f2f3286140e5b2f3dac8e7c02e2a, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:35.082163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:35.083841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:35.088522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.089076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.089543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.095747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 434/891] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:34 2022] GET /identity/v3/role_assignments?user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=ec07f2f3286140e5b2f3dac8e7c02e2a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:35.114559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:35.116327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:35.190498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:35.208333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 458/892] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:35.231980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:35.350111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb158062-68e3-481e-9047-2df7a033dc98 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:35.386922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:35.387789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.388072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.388347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.389015 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:35.390417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:35.403318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:35.404417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:35.405504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 435/893] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:35.415207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:35.524824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8c47b-a2a4-4e41-8ae2-46741de74233 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:35.562347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:35.563337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.563794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.564232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.565126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:35.565579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:35.567142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf5eb-11e0-4c5e-b6d3-c6d6db2d12e5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:35.573152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 459/894] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:35.582397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:35.666108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e68c2-5f44-4876-9301-36ac1e2788c7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.666769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e68c2-5f44-4876-9301-36ac1e2788c7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.667278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e68c2-5f44-4876-9301-36ac1e2788c7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.670394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 460/895] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:35.675114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.675746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.676227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.686510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:35.694457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:35.696146 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:35.750092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:35.763907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db5c13-2f66-4961-9620-cc00e3e36c62 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:35.795514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 461/896] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:35.813500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:35.829097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:35.830646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.831171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.831771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.832724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:35.833352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:35.835442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:35.842216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 436/897] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:35.942335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df0827-6da5-4b75-9fa8-353e97713766 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:35.995025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:35.996700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:35.997376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:35.998132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:35.999723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.002824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:36.027210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:36.029122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:36.030730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:36.040432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 462/898] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:36.047000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:36.206388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c33e0-8396-4e9a-a5e0-216ae7d0cea3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:36.233005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:36.247710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:36.248414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.248700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.248980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.249713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:36.250024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.251461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:36.257654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 437/899] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:36.268731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:36.343376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:36.414785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e9285-cc60-43aa-8db5-a3e21f69d3e2 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:36.453815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:36.454582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.454866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.455193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.456044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:36.456358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.457954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:36.469472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 463/900] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:36.470951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:36.589090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c19c69-2ee7-4641-b4ed-16216dcbd0eb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:36.625929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:36.626606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.626888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.627168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.628017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] RBAC: Adding query filter params (interface=public, service_id=3a9993a9d57444399bc7985bf45ab198, region_id=RegionOne) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:36.628337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.629732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd8cd7-c475-4302-a4fc-8f75ff451d4f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:36.638431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 438/901] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:36 2022] GET /identity/v3/endpoints?service_id=3a9993a9d57444399bc7985bf45ab198&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:36.644092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:36.692981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdabebe-6070-4fc8-b46a-5544e6236800 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.693306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdabebe-6070-4fc8-b46a-5544e6236800 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.693555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdabebe-6070-4fc8-b46a-5544e6236800 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.695291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 439/902] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:36.701205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.701447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.701615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.719627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:36.720767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:36.778854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a2916-1eb6-45ae-9e2d-58f6751a5dac None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:36.780558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:36.793154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:36.814774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 440/903] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:36.825714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:36.826830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.827326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.827822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.828750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.830407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:36.837456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:36.837967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 464/904] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:36 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:02:36.953325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6849677-b08f-49f4-b001-750ee6db8962 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:36.993986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:36.994757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:36.995039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:36.995314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:36.996032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:36.997446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:37.011270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:37.012278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:37.013098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 441/905] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Jul 21 12:02:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:37.020586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.143578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c42ced-38ab-48d5-a27d-a3be96e0b33d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:37.181177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:37.181871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.182159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.182431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.183165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:37.183472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:37.184919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:37.191004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 465/906] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:37.196995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.251203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:37.308523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c8bd-3a50-41de-a8af-bb1d0709e807 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:37.345707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:37.346394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.346732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.347045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.347957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:37.348267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:37.349683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3cdcc-bebf-4eb2-a38d-a02b84c4caaa None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:37.355793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 442/907] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:37.364674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.409803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:37.412021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdfda1-e91f-4628-aa2c-d635df0a0e36 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.412704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdfda1-e91f-4628-aa2c-d635df0a0e36 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.413184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdfda1-e91f-4628-aa2c-d635df0a0e36 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.415276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 443/908] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:37.421093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.421679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.422146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.440720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:37.441822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:37.497665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0c320-6e40-4d47-86f9-a2cba381469b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:37.519947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 444/909] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:37.537804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.539525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:37.540674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.542044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.542529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.543587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] RBAC: Adding query filter params (interface=public, service_id=8cbce5f2bbee427ba8493ebf2a5aeed8, region_id=RegionOne) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:37.544116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:37.545730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:37.555316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 466/910] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/endpoints?service_id=8cbce5f2bbee427ba8493ebf2a5aeed8&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:37.564821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.609694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:37.704045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3807e1-38fd-4cc6-9ec4-e3afb9c74a18 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:37.706122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:37.707171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23f2f4-200a-4540-b368-4b117c750b56 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:37.760667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:37.762396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.762911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.763380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.775401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:37.776495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:37.776968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:37.777451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:37.779100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:37.779245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:37.781935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:37.789760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:37.790759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:37.791617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:37.799519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 467/911] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/services => generated 1268 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:02:37.799882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 445/912] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:37.806933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:37.968655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ce6e5-31da-4caa-9e03-290d2a9321fd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.013886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.014992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.015673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.016176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.017177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:38.017695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.019338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c775ca-f786-43df-b9fc-1ea956d1b994 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.030020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 468/913] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:02:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:38.039716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.113817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7914fb-ca01-4cfc-bda3-d836c7147960 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.114166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7914fb-ca01-4cfc-bda3-d836c7147960 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.114372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7914fb-ca01-4cfc-bda3-d836c7147960 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.116241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 469/914] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:38.122025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.122308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.122590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.141607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:38.142849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:38.170023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:38.195870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:38.200662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819954-cedb-4261-8ea4-c6daa318d791 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.219948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 470/915] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:38.237850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.238663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.238943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.239212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.243441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.251622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.253290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:38.272667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:38.273628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:38.275075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:38.275075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:38.275075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:38.275075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:38.275075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 446/916] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:38.280181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.355670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d0917-4ead-48a8-a9e7-d82cc8638d23 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.391662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188909-70b2-455a-9234-96ff1273ba8e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.393129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.393883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.394232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.394579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.395393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.396932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.415678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 471/917] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:38 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:38.452739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.453811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.454372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.454843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.465738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:38.466306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.468036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.490147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 447/918] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:38.499128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.544036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:38.621030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3d8b-fca9-4e60-bf4d-ffd52c09fb88 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.659968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.660827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.661103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.661425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.670615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.672131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:38.688636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:38.689654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:38.691014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 472/919] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:38.696535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.709802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:38.814144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31331c-61ca-4529-b5f1-2ce4ca69503a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:38.852022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:38.852751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:38.853038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:38.853313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:38.854173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:38.854487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:38.855972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00d0f-902f-4b10-92c8-feac4e638de0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:38.876317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 448/920] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:38.882100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:38.980054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:38.993251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9db8-9dc1-4df6-b89b-6352c519e823 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.036173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.037050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.037329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.037646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.049304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ddca-841f-47cf-9c23-484470912d6c None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.049607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ddca-841f-47cf-9c23-484470912d6c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.049925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112ddca-841f-47cf-9c23-484470912d6c None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.051123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.052792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.055957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 449/921] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:39.057993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.058286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.058555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:39.074228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:39.075943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:39.077626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:39.077626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:39.077626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:39.077626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:39.077626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 473/922] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:39.080689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:39.081304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.081924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:39.162181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.179505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 450/923] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:39.197496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.213605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4543f3-f5bf-4e8d-806b-d6a5369e4a19 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.248319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.257217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.258457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.259046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.259573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.260520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:39.261131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.262849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.282771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 474/924] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:39.284178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.306866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabac087-dc38-47a1-a58e-66c19479846e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.347017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.347792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.348120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.348395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.349068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.350604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:39.363392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:39.364469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:39.365321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 451/925] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:39.371217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.408396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fd2cc-b73c-4514-8578-15795929fc6b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.434532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.452762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.454088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.454586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.455059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.464129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.466070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.481203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e3fe8-f36d-4c2b-96b8-e934a2a511f7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:39.482728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:39.483630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:39.484498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:39.491892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 475/926] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:39.493958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.518842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.519604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.519892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.520165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.521152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:39.521279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.522656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.533433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 452/927] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:39.534463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.555104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.621997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485bbc5-7815-4b00-89dc-1f9328d2fa89 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.653516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad59e89-f728-45ce-aeed-1d122d5e0985 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.666127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.667414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.667981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.668496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.669621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:39.670258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.672199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.682517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.695115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 476/928] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:39.695520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.697073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.697872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.698603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.698741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.699493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:39.699779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.701295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.708043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 453/929] 10.222.0.18 () {56 vars in 1114 bytes} [Thu Jul 21 12:02:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:39.714326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:39.822811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae024eb-b053-40f8-8305-c4de89d053e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.829109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882df8-b2df-459a-8e2d-d801c7d0add7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:39.836775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:39.866720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.867395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.867778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.868064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.868817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.869117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:39.870290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:39.870411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c6af3-7d7b-4857-b31f-fc71d478ace9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.870879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:39.871349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/7da32d4f2caf4f0aa22f839c4a3e6b6b/roles/ec07f2f3286140e5b2f3dac8e7c02e2a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:39.874516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=7da32d4f2caf4f0aa22f839c4a3e6b6b, role_id=ec07f2f3286140e5b2f3dac8e7c02e2a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:39.876790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d2c55-6d7c-4336-afc4-5fc1170fe01f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:39.903604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 454/931] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:02:39.904063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 477/931] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:02:39 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/7da32d4f2caf4f0aa22f839c4a3e6b6b/roles/ec07f2f3286140e5b2f3dac8e7c02e2a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:02:40.610861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50152-2ffe-43e0-8074-fe9adda84e22 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.611640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50152-2ffe-43e0-8074-fe9adda84e22 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.611640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f50152-2ffe-43e0-8074-fe9adda84e22 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.613149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 455/932] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:40.619075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.619867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.619867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.622787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f314094-9a3c-456b-8334-03e8726370a9 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.623072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f314094-9a3c-456b-8334-03e8726370a9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.623351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f314094-9a3c-456b-8334-03e8726370a9 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.625296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 456/933] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:40.633363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.633793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.634174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.643123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:40.644638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:40.653770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:40.655304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:40.698960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:40.751667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 457/934] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:40.753528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 478/935] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:40.772165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:40.777137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:40.891081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:40.897961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a414d3-5b66-4de6-8cac-6eb3dfbaf453 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:40.914888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce0ec1-3ccc-4b5d-ba97-b1144ca40d7d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:40.940859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:40.941904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.942282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.942713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.943553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:40.945083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:40.958964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:40.959223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:40.960749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:40.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:40.963803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:40.964298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:40.964701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:40.965502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:40.967037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:40.969266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 479/936] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:40.970718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:40.982630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:40.983915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 12:02:40.984961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:40.993379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 458/937] 10.222.0.18 () {56 vars in 1132 bytes} [Thu Jul 21 12:02:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:40.994286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.043138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:41.100641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e658f-010d-4cba-8ddb-bf489b8a66f3 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.133301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f7074-2b93-40d3-9fa0-ae68252a8f06 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.137521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.138844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.138844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.138844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.139606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:41.139921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.141410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.147507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 480/938] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:41.156579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.181631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.182621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.183065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.183333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.184263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:41.184718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.186320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.196861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:41.197446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 459/939] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:41.198610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.292009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db9ec-8f9f-45ce-afb1-63d7cfbf93aa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.322788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:41.331879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2de73-9bf9-41f1-aab1-3b1df70285ea None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.339361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.340184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.340537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.340882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.341765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:41.342769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.344301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.355656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 481/940] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:41.359416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.378562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.379633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.380166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.380617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.381630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:41.382118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.383892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.395185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 460/941] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:41.396111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.473516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:41.492872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e42e01-4fb6-4e91-a060-b3a367e88def None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.539724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d6677-bbc5-4ab6-bec2-9f24940f718c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.555373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.556504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.557001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.557477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.558521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] RBAC: Adding query filter params (interface=public, service_id=669e283609ca43a692c64bcf5a86f1e6, region_id=RegionOne) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:41.559049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.560773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.567623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 482/942] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:41 2022] GET /identity/v3/endpoints?service_id=669e283609ca43a692c64bcf5a86f1e6&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:41.575548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:41.603895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.604915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.605337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.605765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.607355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.610537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4192741-6b0a-49d2-ae0b-60ac12e6419f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.647599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 461/943] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:41.706508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786232a9-2d41-43b6-ac8c-d61ecc50a29e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:41.756509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:41.757188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:41.757475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:41.757752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:41.758531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:41.759995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45421615-88ef-4928-a55e-c225a49cab6f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:41.771776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 483/944] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:41 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:02:42.052718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1557e4-4f16-432b-8524-66f69b436e60 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.052981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1557e4-4f16-432b-8524-66f69b436e60 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.053139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1557e4-4f16-432b-8524-66f69b436e60 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.054929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 462/945] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:42.060748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.061149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.061309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.079621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:42.080849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:42.090813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:42.161109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 484/946] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:42.184808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1557e4-4f16-432b-8524-66f69b436e60 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:42.208507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:42.358831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1557e4-4f16-432b-8524-66f69b436e60 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:42.402781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:42.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.403941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.404217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.416804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-7edc4256-ebd1-4007-b752-5a7fb8526fe0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:42.425131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:42.426867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.428494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 463/947] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:02:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:42.437335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:42.470105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d295-5d88-4d6f-b955-6f3756e006c5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.470485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d295-5d88-4d6f-b955-6f3756e006c5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.470659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51d295-5d88-4d6f-b955-6f3756e006c5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.472522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 464/948] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:42.478414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa798-365a-459e-aa35-7712e514f543 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.478768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa798-365a-459e-aa35-7712e514f543 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.478965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa798-365a-459e-aa35-7712e514f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.497807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-330aa798-365a-459e-aa35-7712e514f543 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:42.499067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-330aa798-365a-459e-aa35-7712e514f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:42.583918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 465/949] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:42.602387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aa798-365a-459e-aa35-7712e514f543 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:42.612951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:42.643627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137aa2c2-a74e-4ded-8b7d-4231e7217845 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:42.685155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:42.685846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.686127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.686396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.687161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:42.687464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:42.688933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:42.705634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 485/950] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:02:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:42.706635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:42.713185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:42.802068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aa798-365a-459e-aa35-7712e514f543 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:42.844692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:42.845028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:42.845170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:42.845504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:42.846092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:42.847607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:42.849043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING keystone.server.flask.common [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 12:02:42.870748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.sql.core [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 12:02:42.870748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 12:02:42.870748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [parameters: {'id': '68db11d37e5c4f5fa431d4777267dd77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 12:02:42.870748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80383) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68db11d37e5c4f5fa431d4777267dd77.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68db11d37e5c4f5fa431d4777267dd77. Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:02:42.888866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result.read() Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 12:02:42.890485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application yield resource Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise self.value Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application session.commit() Jul 21 12:02:42.893005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.session.flush() Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application compat.raise_( Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 12:02:42.894768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 12:02:42.898763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result.read() Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:02:42.903424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [parameters: {'id': '68db11d37e5c4f5fa431d4777267dd77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:42.906872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:42.912347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68db11d37e5c4f5fa431d4777267dd77. Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 466/951] 10.222.0.18 () {60 vars in 1140 bytes} [Thu Jul 21 12:02:42 2022] POST /identity/v3/projects => generated 262 bytes in 294 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:42.915918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:42.926634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81343db9-9979-4c6c-8def-279a3d17e10a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:42.969584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:43.001009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.003017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.003886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.004667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.016385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad42dc-5c12-4c22-9e8f-31bfe7c9c2b0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:43.027802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.030999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.057078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.057895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.058165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.058443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.066174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.067929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:43.069217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:43.070976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:43.072869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:43.072869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:43.072869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:43.072869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:43.077129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 486/952] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:43.088636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:43.094126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:43.095868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:43.097679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:43.106223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 467/953] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:43.107334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:43.197288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:43.222474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf8c6d-5b1e-4e0f-8af6-722e839dd738 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:43.249079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:43.261437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.262673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.263414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.264018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.265419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:43.265826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.267431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.292574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fb5fc-d08e-410b-bec8-f825c9f42345 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:43.296305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 468/954] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:43.363637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.365247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.366052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.366755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.368514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:43.369359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.372248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.397976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 487/955] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:43.403736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:43.525866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebcbf5-f208-46f9-892d-1af2bf02d5e3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:43.563812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.564659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.564941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.565216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.574411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.575947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:43.592512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:43.593685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:43.595842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 469/956] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:02:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:43.600982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:43.780488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0da00-32ae-4c2b-b74f-0845d59ff6af None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:43.813449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:43.851453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:43.852759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.853248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.853747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.855460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:43.856047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:43.858854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5992ac7-b0ae-4aee-b46b-31d8ee13b54c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:43.896709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 488/957] 10.222.0.18 () {56 vars in 1149 bytes} [Thu Jul 21 12:02:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:43.902913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:43.966622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:43.983877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a50e-345f-46d2-9bfa-15d930eca3da None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.984084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a50e-345f-46d2-9bfa-15d930eca3da None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.984548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a50e-345f-46d2-9bfa-15d930eca3da None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:43.987659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 489/958] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:43.995959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:43.996237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:43.996511 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.017104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:44.018491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:44.025701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f7688-ec31-4a13-9292-71c45e796e39 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.065503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.066460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.066897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.067332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.111991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 490/959] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:44.141677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:44.143423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:44.160983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.190764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:44.259134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 470/960] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:44.260287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.347424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda313ac-7701-4cd7-ba39-78194c9b4419 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.353137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:44.392691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776184-22cb-49b7-9fbb-e2bee51da282 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.421097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.423203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.424049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.424989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.426690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:44.429736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:44.430559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.431605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.431891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.432172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.443685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:44.445312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:44.447946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:44.461435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:44.463570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:44.465342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:44.466798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:44.476819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:44.480761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:44.480761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:44.480761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:44.480761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:44.480761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 471/961] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:44.481863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 491/962] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:44.482079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.485555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.637018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d164acb-8181-48c7-b96c-ca0b9d0f2982 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.665381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e7e53-5b96-4478-b1f9-fe303b8e44d1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.687162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.689249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.690340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.691953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.694477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:44.696262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:44.700279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:44.714304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 492/963] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:44.721817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.726082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.726869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.727134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.727415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.728247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:44.728554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:44.730005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:44.744680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 472/964] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:44.758175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:44.815951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:44.876394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:44.934597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8da19-8472-48b1-b2d0-6ff848c6e708 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.963436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e2728-b99a-4e85-a57f-21461eb0b4cc None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:44.995110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:44.996667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:44.997378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:44.998128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:44.999710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] RBAC: Adding query filter params (type=compute) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:45.000535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.003320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a487575-3f2d-4d3f-ad02-df231e47acde None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.019800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 493/965] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:45.022387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf8ee-4673-4b31-b810-36d3b316dd0a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.022889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf8ee-4673-4b31-b810-36d3b316dd0a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.023378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf8ee-4673-4b31-b810-36d3b316dd0a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.026506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 494/966] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:45.031916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:45.032392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.032648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.032904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.033151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.034792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.034792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.050122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.053390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.055198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:45.056624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:45.090641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:45.092404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:45.094065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:45.095984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 473/967] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:45.104172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.184716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 495/968] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:45.212914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.242146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74be2f-ddd0-40ee-89cf-68bee607f590 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:45.259752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:45.267667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:45.286893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:45.287620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.287897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.288183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.288989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:45.289348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.290837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.307099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 474/969] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:45.313944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.430349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a46d12-d10f-4163-b31b-b81ed3831972 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:45.502440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:45.504989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.505657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.506688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.508190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.509927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddfc05-67d1-4b4f-9bd6-dafa1b8bee22 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:45.510986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:45.546016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:45.548286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:45.550317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:45.555032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 496/970] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:45.563630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.581046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:45.582192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.583240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.583723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.586653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] RBAC: Adding query filter params (role.id=ec07f2f3286140e5b2f3dac8e7c02e2a, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:45.587216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.591135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41710cd7-68f7-4290-b0fe-1d41cafa67e1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.607673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 475/971] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:45 2022] GET /identity/v3/role_assignments?user.id=7da32d4f2caf4f0aa22f839c4a3e6b6b&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=ec07f2f3286140e5b2f3dac8e7c02e2a => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:45.703939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d1a-50a2-4d96-82b4-b24afa2abde8 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.704215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d1a-50a2-4d96-82b4-b24afa2abde8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.704582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41d1a-50a2-4d96-82b4-b24afa2abde8 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.706399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 476/972] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:45.712763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.713044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.713320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.732922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:45.734689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:45.740404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:45.760425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ced33-fcaf-4545-8c51-8d1948e2ebd9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:45.782608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:45.820155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:45.821333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:45.821621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:45.821896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:45.822629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:45.822940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:45.824573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:45.832277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 497/973] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:45.843469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.850410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 477/974] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:45.880639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:45.980773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24accc29-d670-4bf5-9a29-dc97599e6bcd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:46.019526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2047faf-e330-41c9-922e-b5f9d6f9dabd None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:46.022052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:46.023034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:46.023681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:46.024175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:46.025315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:46.026053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:46.028240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:46.034535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 498/975] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:46.044690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:46.071303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:46.072060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:46.072350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:46.072631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:46.073630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:46.074767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:46.076348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:46.095220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.sql.core [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 12:02:46.095220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 12:02:46.095220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [parameters: {'user_id': '0958fc3bb4e7450186f19a94165c480a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 12:02:46.095220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80383) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result.read() Jul 21 12:02:46.105109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:02:46.106117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application yield resource Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application next(self.gen) Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise self.value Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application session.commit() Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 12:02:46.107093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.session.flush() Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application compat.raise_( Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 12:02:46.108039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 12:02:46.108935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application conn.query(q) Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application result.read() Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 12:02:46.111676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [parameters: {'user_id': '0958fc3bb4e7450186f19a94165c480a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:46.114193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 12:02:46.116213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 478/976] 10.222.0.18 () {60 vars in 1131 bytes} [Thu Jul 21 12:02:45 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:46.118615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:46.157081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:46.229116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0436d2-2df1-4511-a64d-c54b9f6c5e23 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:46.256100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b4af0-9013-474e-b498-d4727a16f006 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:46.293893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:46.295595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:46.296655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:46.297646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:46.299669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:46.300228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:46.300489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:46.300764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:46.301039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:46.304266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87675d08-c6ba-4fb8-a794-9ce1ae8b63e3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:46.309557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:46.311617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:46.329738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:46.332816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 12:02:46.335071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:46.338097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 479/977] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:02:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:46.338097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:46.356447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 499/978] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:02:46.511611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:46.525616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f2fde-95e7-4c74-9b6f-7f9662269ff8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:46.599312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:46.600349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:46.600837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:46.601331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:46.603029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:46.603624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:46.606168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7c4d-7626-4c7d-a6fb-5143a51eee69 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:46.655652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 480/979] 10.222.0.18 () {56 vars in 1101 bytes} [Thu Jul 21 12:02:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:47.493308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77e78-f687-48be-96ff-57302a05f0b7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:47.494075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77e78-f687-48be-96ff-57302a05f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:47.494075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77e78-f687-48be-96ff-57302a05f0b7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:47.495412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 500/980] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:47.500889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2994c-e853-4cbc-89e0-d87c9ba98265 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:47.501605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2994c-e853-4cbc-89e0-d87c9ba98265 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:47.501605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2994c-e853-4cbc-89e0-d87c9ba98265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:47.519882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bfb2994c-e853-4cbc-89e0-d87c9ba98265 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:47.521417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bfb2994c-e853-4cbc-89e0-d87c9ba98265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:47.621258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 481/981] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:47.661663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77e78-f687-48be-96ff-57302a05f0b7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:47.809698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:47.862628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77e78-f687-48be-96ff-57302a05f0b7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:47.920387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:47.921140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:47.921546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:47.921840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:47.923038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:47.925187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5b3ac-53c3-458a-b703-199f1bdd2481 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:47.950538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 501/982] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:47 2022] POST /identity/v3/services => generated 242 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:48.058899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a188-4222-4613-851d-3f5c90e475f8 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.059225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a188-4222-4613-851d-3f5c90e475f8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.059618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a188-4222-4613-851d-3f5c90e475f8 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.061440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 482/983] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:48.067264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.071144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.071144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.089201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:48.090429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:48.206581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 502/984] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:48.230371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a188-4222-4613-851d-3f5c90e475f8 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:48.387613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:48.446182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a188-4222-4613-851d-3f5c90e475f8 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:48.515374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:48.516840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.517282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.517758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.518981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:48.521666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-b56c35c1-3bab-4d64-9ac0-485507fee1e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:48.544528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:48.546485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:48.548177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 483/985] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Jul 21 12:02:48 2022] GET /identity/v3/services/image => generated 86 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:48.553918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:48.657347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0e377-d4ad-4045-aa65-0b22d076867a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.657852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0e377-d4ad-4045-aa65-0b22d076867a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.658832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0e377-d4ad-4045-aa65-0b22d076867a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.661250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 484/986] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:48.663313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:48.668963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.669721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.669917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.692869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:48.694938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:48.730721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38928-17bc-41e9-a7c0-765274ba529d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:48.773258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:48.774202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:48.774639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:48.775096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:48.776122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:48.776572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:48.778328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:48.790062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 503/987] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:48.791395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:48.820436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 485/988] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:48.858632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:48.954736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:48.956339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fdc75b-09c5-4a88-a5ef-aec41ec914d0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:48.992780 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:49.007949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.009043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.009507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.009960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.011266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:49.011829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.013782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a23b8-77e4-44c5-bf62-1b3699ab7056 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:49.014387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.024282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 504/989] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:49.063912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.064763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.065043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.065318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.065997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.067410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:49.081051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:49.083050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:49.085427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:49.088088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 486/990] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:49.092138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:49.220852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae6d83-9759-48bd-944d-ab01e3f8e6e6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:49.263588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.264534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.264964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.265379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.266243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:49.266695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.268309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.279369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 505/991] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:49.281239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:49.408416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb74cb-1738-4aab-8dde-7928248d78bb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:49.441048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:49.448206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.449057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.449465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.449878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.450736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:49.451175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.452814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.464477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 487/992] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:49.466312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:49.528435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:49.585901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba26ed1-fccc-4c8f-8b95-312f29184e88 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:49.626171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.627027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.627439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.627869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.628816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] RBAC: Adding query filter params (interface=admin, service_id=669e283609ca43a692c64bcf5a86f1e6, region_id=RegionOne) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:49.629249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.630790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc868-16dd-48fd-9eb8-1ca6c677a044 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.643928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 506/993] 10.222.0.18 () {56 vars in 1249 bytes} [Thu Jul 21 12:02:49 2022] GET /identity/v3/endpoints?service_id=669e283609ca43a692c64bcf5a86f1e6&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:49.644168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:49.760235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79d818-0cf3-4fd9-85f4-0e223554f0c7 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:49.797078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:49.797670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:49.797946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:49.798222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:49.798918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:49.800357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86d4a7-acbd-498b-b829-d7c7140b16d2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:49.807832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 488/994] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:49 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:02:50.304277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634a332-00d6-4e94-835f-0094766469d0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:50.304618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634a332-00d6-4e94-835f-0094766469d0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:50.304937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634a332-00d6-4e94-835f-0094766469d0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:50.306350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 507/995] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:50.312068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:50.312348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:50.312642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:50.331519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:50.332731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:50.398992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:50.418546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 489/996] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:50.442161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634a332-00d6-4e94-835f-0094766469d0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:50.625457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634a332-00d6-4e94-835f-0094766469d0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:50.656852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:50.691568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:50.692885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:50.693379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:50.693847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:50.695104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:50.697878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:50.721315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:50.723179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:50.733268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 508/997] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:50.734828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:50.917168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3482b-eba8-4130-83f9-ad3ea9c6ea3d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:50.973818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:50.974663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:50.975168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:50.975710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:50.977013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:50.977528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:50.980169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:50.994979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 490/998] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:50.996346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:51.118374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74203ba-1720-468e-9579-44b26fb7374f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:51.181259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:51.182305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:51.182751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:51.183224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:51.184634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:51.185073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:51.187563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:51.196976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 509/999] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:51.203508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:51.309208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:51.388827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bacf-2aed-487c-8c6b-0d41c204dd6f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:51.450389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:51.451422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:51.451951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:51.452438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:51.453990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] RBAC: Adding query filter params (interface=public, service_id=6022369a30ad4c47970347a106757f27, region_id=RegionOne) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:51.454448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:51.456741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22104592-0447-4ebf-bdc7-c83f245d872d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:51.467709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 491/1000] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:51 2022] GET /identity/v3/endpoints?service_id=6022369a30ad4c47970347a106757f27&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:51.472971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:51.526046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:51.586627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b752bcb-4f92-4d2f-a1dd-47765a2a434e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:51.625905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:51.626597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:51.626846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:51.627104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:51.627834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:51.629218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8bfc22-75af-4ca0-8f5e-b72ff22cfdce None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:51.636622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 510/1001] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:51 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:02:52.007403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27c6-5eab-4bae-8a1c-0f726b6ac594 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.008255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27c6-5eab-4bae-8a1c-0f726b6ac594 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.008255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27c6-5eab-4bae-8a1c-0f726b6ac594 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.009499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 492/1002] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:52.015815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.016137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.016516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.029404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280c4d3-0042-4671-adda-72d464968b31 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.029688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280c4d3-0042-4671-adda-72d464968b31 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.029956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280c4d3-0042-4671-adda-72d464968b31 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.031443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 493/1003] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:52.037360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.037714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.037859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:52.038007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.039458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:52.056884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:52.058131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:52.161167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 494/1004] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:52.173698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 511/1005] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:52.184506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:52.211882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:52.256753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:52.274826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:52.328141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60bc2-79a7-44fd-bb1e-c1c8cdee6b9a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:52.373574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471284cf-5ccd-408f-ade1-c324f71708dc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:52.380837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:52.381661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.381936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.382205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.382953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:52.384405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:52.398199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:52.401145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:52.402815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 495/1006] 10.222.0.18 () {56 vars in 1126 bytes} [Thu Jul 21 12:02:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:52.406283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:52.424084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:52.424674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.424942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.425215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.426048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:52.427573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:52.455520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 512/1007] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:52 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:52.542847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28621762-7456-4b3e-80da-9203ba83b469 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:52.564369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:52.606562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:52.607678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.607970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.608233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.609041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:52.609346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:52.610863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:52.619415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 496/1008] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:52.627166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:52.759467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:52.767956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade4b8-f5e7-4437-9a36-affdefb9338f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:52.809237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:52.809828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.810112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.810356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.811165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:52.811461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:52.812925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e8ca-b96c-4fe1-8a0a-adc06b1aec18 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:52.823525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 513/1009] 10.222.0.18 () {56 vars in 1122 bytes} [Thu Jul 21 12:02:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:52.826289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:52.946409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd245ae6-ff4d-42de-9949-f0dd94c6355a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:52.986327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:52.987176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:52.987636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:52.988057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:52.988955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:52.992714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a903321-2d3a-46d1-9c94-8ea641830769 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:53.019535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 497/1010] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:53.818517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119adb-fab7-4a6f-88a9-9f15bfaa9d4d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:53.819136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119adb-fab7-4a6f-88a9-9f15bfaa9d4d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:53.819136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d119adb-fab7-4a6f-88a9-9f15bfaa9d4d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:53.820751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 514/1011] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:53.826117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:53.826371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:53.826680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:53.844911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:53.846115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:53.874775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:53.928719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 498/1012] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:53.952643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119adb-fab7-4a6f-88a9-9f15bfaa9d4d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:54.084913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d119adb-fab7-4a6f-88a9-9f15bfaa9d4d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:54.124355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:54.125248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.125591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.125927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.126663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:54.128172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:54.141225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:54.142422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 12:02:54.143354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.144458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 515/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:54.155002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:54.275802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26418f5-6e53-4745-8838-720022e08c89 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:54.317039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:54.317727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.318015 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.318338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.319087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:54.319399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:54.320902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:54.327281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 499/1014] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:54.333302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:54.369935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:54.499099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda6297-41e9-4497-907b-2ee65975d627 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:54.534568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:54.535250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.535538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.535839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.536551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:54.536854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:54.538354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9587b-4f5e-404d-b4e3-d1c5298c1380 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:54.547882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 516/1015] 10.222.0.18 () {56 vars in 1112 bytes} [Thu Jul 21 12:02:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:54.549971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:54.575385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:54.603045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df45d-c010-4356-b5e7-734e5cace2ff None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.603327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df45d-c010-4356-b5e7-734e5cace2ff None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.603641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df45d-c010-4356-b5e7-734e5cace2ff None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.605191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 517/1016] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:54.612434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.612712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.612986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.630163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:54.631264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:54.667102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7d9ea-209c-4865-8d71-3a150b8ef26f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:54.704978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 518/1017] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:54 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:54.716964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:54.717873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.718355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.718769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.719736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:54.721303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:54.728480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:54.743676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 500/1018] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:54.766018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:54.871334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9daf9d-f41f-4842-88c5-0a1e9cb5cbc7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:54.921208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:54.921544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:54.921758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:54.922037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] PATH_INFO: `/v3/services/image` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:54.922699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:54.924143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:54.938935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:54.940035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:54.940853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:54.942686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 519/1019] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Jul 21 12:02:54 2022] GET /identity/v3/services/image => generated 86 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:54.946894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.038708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:55.068961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e0fc0-4009-4603-aa4b-e95432faba55 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.109765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.110450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.110821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.110996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.112039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] RBAC: Adding query filter params (name=image) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:55.112039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:55.113421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:55.123563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 501/1020] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:55.125987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.231530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:55.261086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b263f-2712-45c9-acca-6b36440f599e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.307437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.308558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.308907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.309271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.310428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] RBAC: Adding query filter params (type=image) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:55.310945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:55.313255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f767ad-db05-457e-86c3-b16702cfb4cb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:55.321346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 520/1021] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:55.329221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.406970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078da6b7-d474-46cf-92b3-556e8692f4d4 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.407311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078da6b7-d474-46cf-92b3-556e8692f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.407580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078da6b7-d474-46cf-92b3-556e8692f4d4 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.409088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 521/1022] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:55.414519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.414816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.415084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.432573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:55.433779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:55.471245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3a945-00f8-4e83-87c9-07a078a65457 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.520999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.522079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.522538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 522/1023] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:55.522748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.523039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.524026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] RBAC: Adding query filter params (interface=public, service_id=0cefea01f14644eb929307cadc28f389, region_id=RegionOne) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:55.524342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:55.525777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:55.536405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 502/1024] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:02:55 2022] GET /identity/v3/endpoints?service_id=0cefea01f14644eb929307cadc28f389&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:55.544919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.556556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.572405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:55.648291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:55.688330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45aa46-2ff1-4a30-80ad-20959e3eeb65 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.698630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a565818-c7ea-4dd3-be9f-037f34fb782e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.735462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.736165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.736448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.736721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.737462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:55.738906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:55.744263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.745632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.754884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.755093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 523/1025] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:02:55 2022] GET /identity/v3/services => generated 1957 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:02:55.755175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:55.755175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:55.755175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5da019-9aa3-45ea-aa0c-efe7122dbf7b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:55.755175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 503/1026] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:55.763172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:55.937118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47b7b3-2e3a-41e0-8335-93c278d18ec6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:55.996975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:55.998090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:55.998574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:55.999071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:56.000442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:56.002409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00999aeb-6528-432c-9194-c01c2122f00b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:56.013820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:56.043330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 524/1027] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:02:55 2022] POST /identity/v3/projects => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:02:56.940155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda34e1-72e4-4a64-9e73-fccc17fd28d6 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:56.940841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda34e1-72e4-4a64-9e73-fccc17fd28d6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:56.940841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda34e1-72e4-4a64-9e73-fccc17fd28d6 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:56.942226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 504/1028] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:56.948868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:56.949457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:56.949457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:56.967461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:56.968691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:57.038715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 525/1029] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:57.078440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda34e1-72e4-4a64-9e73-fccc17fd28d6 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:57.215554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:57.259985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda34e1-72e4-4a64-9e73-fccc17fd28d6 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:57.319304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:57.320792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.321287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.321746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.343264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:02:57.359152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:57.360470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:02:57.361455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:57.363836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 505/1030] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:02:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:57.370117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:57.492736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4fde2-58f5-47ac-8ce0-0f6311fab064 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:57.517128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:57.537655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:57.538360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.538642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.538913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.539817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:57.540131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:57.541587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:57.558568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 526/1031] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:02:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:57.564781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:57.681149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62663666-44b8-42a7-88c0-3e9a246b16d8 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:57.721208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:57.722067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.722327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.722774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.724894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:57.736265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:57.737932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-17861e9f-a31e-4917-9811-22dc4b72431f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:57.768967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:57.770099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:57.771360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:57.771360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:57.771360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:57.771360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:57.772480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 506/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:02:57.779661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:57.843309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faadc1-2a29-4b40-850b-c101413f2f9e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.843730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faadc1-2a29-4b40-850b-c101413f2f9e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.843929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faadc1-2a29-4b40-850b-c101413f2f9e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.846617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 507/1033] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:02:57.850929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.851249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.851571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.878572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:57.880007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:57.910051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f563-28d5-4298-8825-c56744e3d009 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:57.953999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:57.955032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:57.955032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:57.955369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:57.956015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:57.956327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:57.957807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:57.977418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 527/1034] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:57.978752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:57.983972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 508/1035] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:02:58.014097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.039066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.128305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c64dd-a8fc-40c1-8c49-8d1bd7059129 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.139016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400703b-bf24-4fc7-8be5-9d01befea00f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.159834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.182386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.183201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.183467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.183777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] PATH_INFO: `/v3/services/image` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.184449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.185852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.186122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.186984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.187411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.187714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.197789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:58.198336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:58.200783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 12:02:58.202257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:02:58.205028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.206070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 509/1036] 10.222.0.18 () {56 vars in 1105 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:58.206070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:02:58.220757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:58.222616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:58.224513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 528/1037] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:02:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:58.228777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.344209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce499b-9b0b-443a-8ec2-68e5a5d67357 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.345837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.364526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a020f-d587-44be-9d15-5950f6f34afe None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.392779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.393512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.393795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.394037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.394921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:58.395246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.396810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.409877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.411098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.411614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.412147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.413179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] RBAC: Adding query filter params (name=image) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:58.413848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.415414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.418729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 529/1038] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:58.429417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 510/1039] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:58.429724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.430722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.500009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.545039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45fc4a-2e12-46cf-8af0-923e25c7d6ed None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.571075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f7188-c4a8-4aa4-aa33-57bc5216ffdb None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.593422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.594519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.594958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.595435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.606520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.607200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.607510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.607775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.608494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] RBAC: Adding query filter params (type=image) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:58.608800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.610191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c484e-1eaa-4224-a002-048fddd6cf4c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.616267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 511/1040] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:58.618020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413764f8-414c-4802-bd77-71006214aa80 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.618457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413764f8-414c-4802-bd77-71006214aa80 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.618934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413764f8-414c-4802-bd77-71006214aa80 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.620766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 512/1041] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:02:58.624158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413764f8-414c-4802-bd77-71006214aa80 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.635395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.695393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.696893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.749206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413764f8-414c-4802-bd77-71006214aa80 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:58.759208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.785075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:58.785751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.786027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.786303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.787056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:58.788495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b8d0b-a1a7-4009-87ce-86f6e118755f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:58.821487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 513/1042] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:02:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:02:58.821487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:58.821487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:58.821487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:58.822135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 530/1043] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:02:58.829361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.842456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:02:58.844222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:02:58.919073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:58.951702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 514/1044] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:02:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:02:58.978706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:58.984469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df32f7-baf4-462c-95c8-0747a3c6a192 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.026803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.028178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.028507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.029476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.050858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.053002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:02:59.072775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:59.073808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:59.074967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:02:59.074967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:02:59.074967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:02:59.074967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:59.074967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 531/1045] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:59.079881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:59.103333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:59.115092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e561a-91c1-4089-8302-ebfbdb81263c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.166280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.167505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.167803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.168081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.169399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.170823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.172749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 515/1046] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:02:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:59.181701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:59.215158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281c1b0-2b0d-47ed-a9c2-5337149cc8f8 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.231668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:59.256357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.257156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.257504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.257817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.258569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:59.258880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.260387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.273182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 532/1047] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:02:59.279007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:59.347203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dcc6-2dde-435a-b3a0-1c3f7580f505 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.402758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.403858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.404321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.404748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.405955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.408570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.433988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b976a-ed47-4797-a000-69caef9f5479 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.439764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 516/1048] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:02:59 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:02:59.500061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.501275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.501677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.502074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.514843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.517445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:02:59.542599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:02:59.543965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:02:59.544868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:02:59.545308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 533/1049] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:02:59.551564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:59.572800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:59.690070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a79cb-11d7-4cff-ab17-10928a429ee1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.737558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.738555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.738829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.739100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.739957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:59.740264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.741745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f38f-9d72-42ca-948a-119e17d29033 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:02:59.759735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 517/1050] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:02:59.767142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:02:59.815363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:02:59.938227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8c83b-dda6-475b-b31b-367f881d0006 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:02:59.988381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:02:59.989346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:02:59.989471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:02:59.989882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:02:59.990892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] RBAC: Adding query filter params (role.id=ec07f2f3286140e5b2f3dac8e7c02e2a, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:02:59.991203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:02:59.993267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124cc75-24af-48b6-8680-ead02c79d257 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:00.007521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 534/1051] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:02:59 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=ec07f2f3286140e5b2f3dac8e7c02e2a => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:00.928092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ab9f-ebca-4bff-90f0-6df25665a607 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:00.928904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ab9f-ebca-4bff-90f0-6df25665a607 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:00.928904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ab9f-ebca-4bff-90f0-6df25665a607 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:00.930338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 518/1052] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:03:00.935757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:00.936071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:00.936354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:00.953770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:00.954983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:01.026336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 535/1053] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:01.049515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ab9f-ebca-4bff-90f0-6df25665a607 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:01.133697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:01.178529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ab9f-ebca-4bff-90f0-6df25665a607 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:01.236986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:01.237999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:01.238427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:01.238855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:01.360608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:01.363020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c496cab-3fed-4d7e-93dd-14ee76ac3629 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:01.471169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:01.503033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 519/1054] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 12:03:01.509045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:01.538554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:01.622549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a546f-cc51-4b1c-a015-60328afb9b20 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:01.639250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d298e-512f-4034-92d3-8779029bda55 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:01.639583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d298e-512f-4034-92d3-8779029bda55 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:01.639926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d298e-512f-4034-92d3-8779029bda55 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:01.641417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 520/1055] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:03:01.646791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:01.647140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:01.647380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:01.663433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:01.664328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:01.664600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:01.664868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:01.665941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:01.667163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:01.676904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:01.678393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:01.698801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:01.700103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:01.701095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:03:01.701095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:01.701095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:01.701095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:01.702755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 536/1056] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:01.710099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:01.786563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 521/1057] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:01.796047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:01.824587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d3423-c94f-49df-9447-8d36cb5a727d None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:01.826769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:01.866523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:01.867252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:01.867545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:01.867808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:01.868549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:01.868847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:01.870237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:01.884436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 537/1058] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:02.007353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c28c4-b1b6-4a7c-b60a-8e5e8392e293 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.010539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.060418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.061133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.061411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.061679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.074310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ff4fcf73-d4e8-4b43-930b-4560593d8534 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:02.082125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:02.084268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:02.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 522/1059] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:02.090581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.227525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50db3-9b4c-4bbf-82bf-7173ce292207 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.263819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a029-41d6-4b5a-a5cc-04fb5a802215 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.264100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a029-41d6-4b5a-a5cc-04fb5a802215 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.264381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a029-41d6-4b5a-a5cc-04fb5a802215 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.265075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.265778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 523/1060] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:03:02.265909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.266198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.266474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.267750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:02.268260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:02.270048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:02.271267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.271588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.271869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.276612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.284729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 538/1061] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:02.292229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.293540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:02.294761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:02.374623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 524/1062] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:03:02.406973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.411247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c8498-b25e-4783-b470-ecfeff454e47 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.451007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.452134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.452500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.452924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] PATH_INFO: `/v3/users/4dc2dad7233c49768448f166caefee30` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.469670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] RBAC: Authorizing `identity:get_user(user_id=4dc2dad7233c49768448f166caefee30)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:02.471164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:02.477897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 539/1063] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/users/4dc2dad7233c49768448f166caefee30 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:02.479059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.511806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.522230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd3dd-babb-4aba-9a6d-4589be376f6a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.553189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.579019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.579878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.580160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.580438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.594181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:02.601991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288768c8-67ba-4b3a-93b4-b282efbbd515 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:02.602562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:02.603439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:03:02.604321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:02.607869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 525/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:02.610055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.653607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.654296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.654651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.655014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.720731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752006b2-a709-4b52-9d4a-b64fd8b42791 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.739774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:02.741916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:02.768342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.769391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.769833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.770181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.771422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:02.771943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:02.774353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:02.788178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 526/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:02.797125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.823793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.827435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:02.878206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 540/1066] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 12:03:02.884691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:02.926015 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9337-b9f2-4f48-a2d8-1f5997e381ad None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.964058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:02.964863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:02.965153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:02.965429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:02.976707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:02.978182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:02.990798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdf26c-8f14-4377-abe2-7b5b6e8ea366 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:02.996637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:02.997560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:02.998625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:03:02.998625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:02.998625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:02.998625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:02.998625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 527/1067] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:03.004241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.031439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.032347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.032628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.032909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.035009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.037179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.038993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 541/1068] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:02 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:03.045882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.079017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:03.092168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:03.128491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74b276-06c9-49a0-9718-be24919f6bdd None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.174665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.175339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.175624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.175924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.176730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:03.176947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.177548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040ad78-663c-49d9-9775-3aea13abfb99 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.178356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.191413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 528/1069] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:03.201091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.223795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.224451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.224725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.224992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.225827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=369cf41efd1948228e4be5990028ecef, user.id=4dc2dad7233c49768448f166caefee30) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:03.226129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.227671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.233745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 542/1070] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/role_assignments?user.id=4dc2dad7233c49768448f166caefee30&scope.project.id=369cf41efd1948228e4be5990028ecef&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:03.328376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd827e7-7e46-446d-9ea9-1cbcac2a74a2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.361296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:03.371816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.372692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.372976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.373248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.382233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.383908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:03:03.401142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:03.402356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:03.403660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 529/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:03.414760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.541694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:03.555711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f86cf4-f6de-48ad-89ea-7a32f80656cc None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.593303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.593979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.594260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.594526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.595371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:03.595700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.597175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.617199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 543/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:03.623278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.755502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0126dcd-2f05-492d-a9dd-2887238635ec None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.791323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.792252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.792450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.792724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.804127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.805673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:03.824393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:03.826296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:03.828099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:03:03.828099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:03.828099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:03.828099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:03.828099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 530/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:03.832853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:03.940292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9734c-b60a-4b09-ba42-4f1ddd63347b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:03.982819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:03.983518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:03.983802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:03.984080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:03.984829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:03.985136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:03.986554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:03.999858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 544/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:04.006496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:04.053521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:04.117341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3e780-a02a-4da7-88ed-7434c48dc90a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:04.157833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:04.158663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:04.158940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:04.159206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:04.166647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:04.168148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:03:04.182226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:04.184167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:03:04.185958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:04.187654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 531/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:04.189620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:04.202966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:04.297991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c60504-3a80-4359-9b17-536e675de3fe None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:04.333718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:04.334675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:04.334948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:04.335214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:04.336100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:04.336405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:04.337858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bd7c9-422d-4aad-8757-14ec11f5f3c4 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:04.352927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 545/1076] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:04.359816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:04.485518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:04.489294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be8ba-c44a-4eaa-8636-31c95a16f463 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:04.528684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:04.529592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:04.529873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:04.530146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/ec07f2f3286140e5b2f3dac8e7c02e2a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:04.532485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=bccc158b75eb41ca8431bbb67af65260, role_id=ec07f2f3286140e5b2f3dac8e7c02e2a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:04.533944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650e933-af5f-4fa5-92e0-1a39726c8b19 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:04.549546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 532/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:03:04 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/bccc158b75eb41ca8431bbb67af65260/roles/ec07f2f3286140e5b2f3dac8e7c02e2a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:03:05.280380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6bcbb-b3d0-48e7-965d-c6c0ad4a2272 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:05.281221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6bcbb-b3d0-48e7-965d-c6c0ad4a2272 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:05.281381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6bcbb-b3d0-48e7-965d-c6c0ad4a2272 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:05.282802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 546/1078] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:05.288749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:05.289361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:05.289361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:05.307140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:05.308382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:05.379348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 533/1079] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:05.402265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6bcbb-b3d0-48e7-965d-c6c0ad4a2272 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:05.510683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:05.555441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6bcbb-b3d0-48e7-965d-c6c0ad4a2272 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:05.595072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:05.595922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:05.596197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:05.596462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:05.607689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:05.615126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:05.616708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:05.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 547/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:05.622556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:05.754237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:05.767341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c8baf-ecb3-4fe7-8239-d320fc422d01 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:05.808460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:05.810249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:05.810537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:05.810859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:05.811675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:05.811986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:05.813399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:05.822538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 534/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:05.829299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:05.964175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f8060-fd39-436c-b2f3-1ad49e97864e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:05.991417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:06.005533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:06.006435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.006717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.007001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] PATH_INFO: `/v3/users/4dc2dad7233c49768448f166caefee30` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.008241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] RBAC: Authorizing `identity:get_user(user_id=4dc2dad7233c49768448f166caefee30)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:06.009817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:06.011743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 548/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:05 2022] GET /identity/v3/users/4dc2dad7233c49768448f166caefee30 => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:06.018042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:06.125936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f433380-f63a-44f5-8d3d-b3052049ad1c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:06.167669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:06.168560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.168835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.169109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.170341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:06.171810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db298a-09aa-4090-aaa5-2990c92dcc39 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:06.173565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 535/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:06 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:06.180126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:06.290633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567fa0b-0909-4d8d-a3e0-4d51d782d45b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:06.328747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:06.329639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.329914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.330186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef/users/4dc2dad7233c49768448f166caefee30/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.332185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] RBAC: Authorizing `identity:create_grant(project_id=369cf41efd1948228e4be5990028ecef, user_id=4dc2dad7233c49768448f166caefee30, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:06.333647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31434d-e29a-4923-8d9e-31b52d1df018 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:06.348570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 549/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:03:06 2022] PUT /identity/v3/projects/369cf41efd1948228e4be5990028ecef/users/4dc2dad7233c49768448f166caefee30/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:03:06.611424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212afd-52ca-45aa-a517-6e6522fd69ec None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.611967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212afd-52ca-45aa-a517-6e6522fd69ec None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.611967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212afd-52ca-45aa-a517-6e6522fd69ec None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.613510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 536/1085] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:06.619441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.619841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.620145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.638117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:06.639350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:06.693619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:06.713042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 550/1086] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:06.736792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27212afd-52ca-45aa-a517-6e6522fd69ec None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:06.761656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:06.847031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27212afd-52ca-45aa-a517-6e6522fd69ec None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:06.882395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:06.883553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:06.883857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:06.884136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:06.899913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:06.908274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:06.910139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:06.911765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 537/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Thu Jul 21 12:03:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:06.918278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:07.055365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d93c6e-31f7-42d7-a098-9d24021ea493 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:07.095933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:07.096632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:07.096918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:07.097198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:07.097968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:07.098268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:07.099732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:07.109299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 551/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:03:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:07.116161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:07.220704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:07.248511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9ef9a-3e22-4794-a249-937db06485ee None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:07.303000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:07.303862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:07.304134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:07.304437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:07.315785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:07.317260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:07.335276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:07.336787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:07.337876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:03:07.337876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:07.337876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:07.337876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:07.337876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 538/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:07.345482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:07.404427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:07.468383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91db3-ef0c-480e-9d0a-6d11c68a2221 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:07.507635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:07.508447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:07.508808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:07.509185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:07.510062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:07.510455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:07.512092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:07.526704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 552/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:07.532552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:07.662708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c092c-ff7c-48e0-b247-5d5300866a0f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:07.693185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:07.699421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:07.700480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:07.700794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:07.701092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:07.721501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:07.724151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:03:07.738918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:07.740422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:07.741449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 539/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Thu Jul 21 12:03:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:07.746414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:07.862974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd214f-2d1a-4f1e-bf38-0ab204f75f05 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:07.873879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:07.901346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:07.902034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:07.902303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:07.902571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:07.903434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:07.903774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:07.905260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:07.926343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 553/1092] 10.222.0.18 () {56 vars in 1133 bytes} [Thu Jul 21 12:03:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:07.932342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.063748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d35c9-17c7-4d50-a154-06966764a425 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.121566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.122689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.123088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.123574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.235637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:08.263263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:08.264814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef57ec7-1bc7-4ab5-865c-61a5b5503835 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:08.390190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 540/1093] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:08.396804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.456067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf00e9d-e09f-4c4d-9689-ac4e3f8b9814 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.456321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf00e9d-e09f-4c4d-9689-ac4e3f8b9814 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.456620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf00e9d-e09f-4c4d-9689-ac4e3f8b9814 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.458305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 541/1094] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:03:08.466041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.466373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.466764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.488937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:08.490057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:08.521845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b7ac7-ebca-401a-baf8-4522f54d3e28 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.564899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 542/1095] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:03:08.569326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.570143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.570420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.570687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.581919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:08.583415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:08.594691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:08.602922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:08.604106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:08.605865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:03:08.605865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:08.605865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:03:08.605865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:08.605865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 554/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:08.610080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.630405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:08.637501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:08.726863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fd64d-3f83-40a4-a420-32c7ba3bd101 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.736032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b43955-939e-4c5f-a0f1-9a27356efcdb None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.771101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.771699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.771968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.772258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.772997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:08.773437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:08.774775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:08.775037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.775900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.776184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.776675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.787376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 555/1097] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:08.789677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:08.795721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:08.797413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:08.799341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:08.800842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 543/1098] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:03:08.804763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.888000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:08.902432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:08.919356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc8360-004f-45d8-a8d3-e29bb583eb55 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.929665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a4958-f032-4d9f-8e21-4920eb8f06d1 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:08.957694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.958513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.959153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.959421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.966939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:08.968453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:03:08.984271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:08.985247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:08.986117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 556/1099] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:08.986773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:08.987921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:08.988420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:08.988912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:08.990551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:08.990924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:08.991416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:08.993463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.008522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 544/1100] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:09.015235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:09.104821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbc971-4aed-4f22-ac7c-c0fb5c62e436 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.126378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4d988-7e0d-4d0b-a920-756920bd1323 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.141661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:09.142329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:09.142600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:09.142866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:09.143703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:09.148249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:09.148249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.156076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:09.162987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 557/1101] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:09.171828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:09.180039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:09.181588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:09.182096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:09.182719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] PATH_INFO: `/v3/users/4dc2dad7233c49768448f166caefee30` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:09.184767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] RBAC: Authorizing `identity:get_user(user_id=4dc2dad7233c49768448f166caefee30)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:09.187555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.190543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 545/1102] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:09 2022] GET /identity/v3/users/4dc2dad7233c49768448f166caefee30 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:09.196751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:09.216618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:09.291700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50861-9c3c-49fd-bf24-9a4a4286fe43 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.325134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593060e-4f1d-4775-8a94-f2f9800f74fe None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.332145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:09.332836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:09.333115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:09.333389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:09.334624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] RBAC: Adding query filter params (role.id=ec07f2f3286140e5b2f3dac8e7c02e2a, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=bccc158b75eb41ca8431bbb67af65260) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:09.334933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:09.336549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.343326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 558/1103] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:09 2022] GET /identity/v3/role_assignments?user.id=bccc158b75eb41ca8431bbb67af65260&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=ec07f2f3286140e5b2f3dac8e7c02e2a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:09.386128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:09.386725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:09.386998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:09.387268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:09.493387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:09.496338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.526182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:09.648054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 546/1104] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 12:03:09.658728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:09.746379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:09.797557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d485e-3233-4a19-a4bd-6490d4d18b1f None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.832977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:09.834207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:09.834524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:09.834805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:09.836140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:09.837594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1711e-0f67-46f6-9a98-62c4787d2668 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:09.839664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 559/1105] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:09 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:09.846061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:09.949077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d8f7-60aa-4d0c-b9fe-cd345f2fac78 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:09.967562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:10.005698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:10.006366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:10.006641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:10.006907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:10.007818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=369cf41efd1948228e4be5990028ecef, user.id=4dc2dad7233c49768448f166caefee30) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:10.008596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:10.009636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeb278-3553-415d-bb9f-856042d64b8e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:10.016918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 547/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:09 2022] GET /identity/v3/role_assignments?user.id=4dc2dad7233c49768448f166caefee30&scope.project.id=369cf41efd1948228e4be5990028ecef&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:12.068856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7495a-93c8-4559-969c-e599da2ce124 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:12.069458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7495a-93c8-4559-969c-e599da2ce124 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:12.069458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7495a-93c8-4559-969c-e599da2ce124 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:12.070791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 560/1107] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:12.076609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:12.077066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:12.077155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:12.095061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:12.096330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:12.165334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 548/1108] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:12.190576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7495a-93c8-4559-969c-e599da2ce124 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:12.302584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7495a-93c8-4559-969c-e599da2ce124 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:12.333333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:12.340162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:12.340967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:12.341247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:12.341518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:12.342803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:12.344281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c1006-4cb7-444a-b45c-e021649a92fb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:12.345918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 561/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:12.352061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:12.459704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0760-28a6-4245-9c0f-f5c9cb1ce7d4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:12.467569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:12.497049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:12.497706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:12.498040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:12.498317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:12.499038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:12.500501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b569a-4901-4008-abb2-a3f857a74d2c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:12.522229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 549/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:03:12 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:03:14.608824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56c203-1c5e-4c63-b2bf-8dfb1c8e436f None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:14.609408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56c203-1c5e-4c63-b2bf-8dfb1c8e436f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:14.609408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56c203-1c5e-4c63-b2bf-8dfb1c8e436f None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:14.610650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 562/1111] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:14.616459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:14.616971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:14.617068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:14.635497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:14.636775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:14.712062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 550/1112] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:03:14.736548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56c203-1c5e-4c63-b2bf-8dfb1c8e436f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:14.842856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56c203-1c5e-4c63-b2bf-8dfb1c8e436f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:14.878039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:14.878826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:14.879107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:14.879388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] PATH_INFO: `/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:14.881092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] RBAC: Authorizing `identity:get_role(role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:14.882555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:14.883978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 563/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:14 2022] GET /identity/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:14.890240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:14.985753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:14.995803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c7349-0b3e-4335-9904-9343c5fdfb4b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:15.030012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:15.030859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:15.031168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:15.031437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] PATH_INFO: `/v3/users/5eeaca1f929d483b868965b6adddc8fd` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:15.044998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] RBAC: Authorizing `identity:get_user(user_id=5eeaca1f929d483b868965b6adddc8fd)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:15.046456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:15.048145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 551/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:14 2022] GET /identity/v3/users/5eeaca1f929d483b868965b6adddc8fd => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:15.054198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:15.121383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:15.159734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ae71f-3f86-4ba0-9f48-d988e4c7db81 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:15.195982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:15.196647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:15.196923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:15.197195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:15.264247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:15.265706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:15.331538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:15.368137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 564/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:15.374356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:15.476786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbdae5-9bd4-476d-9555-afc45953be00 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:15.511636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:15.512503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:15.512780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:15.513056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:15.514251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:15.515717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b473f-55c5-4f82-a1c4-bcab1aed24ff None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:15.517422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 552/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:15 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:15.523970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:15.629222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308092c-d561-4c57-882b-b2a59b651250 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:15.664532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:15.665187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:15.665468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:15.665742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:15.666581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=369cf41efd1948228e4be5990028ecef, user.id=5eeaca1f929d483b868965b6adddc8fd) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:15.666897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:15.668380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee49e54-7740-44b0-8876-3c9fa02e298c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:15.674112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 565/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:15 2022] GET /identity/v3/role_assignments?user.id=5eeaca1f929d483b868965b6adddc8fd&scope.project.id=369cf41efd1948228e4be5990028ecef&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:17.681270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84441b-1ce8-4504-84ad-883a907a59f1 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:17.681847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84441b-1ce8-4504-84ad-883a907a59f1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:17.681847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84441b-1ce8-4504-84ad-883a907a59f1 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:17.683202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 553/1118] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:17.688736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:17.689186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:17.689276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:17.706524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:17.707762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:17.732381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:17.779420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 566/1119] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:17.802330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84441b-1ce8-4504-84ad-883a907a59f1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:17.823467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:17.909446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84441b-1ce8-4504-84ad-883a907a59f1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:17.944465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:17.945322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:17.945602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:17.945876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] PATH_INFO: `/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:17.947399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] RBAC: Authorizing `identity:get_role(role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:17.948815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:17.950240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 554/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:17 2022] GET /identity/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:17.958545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:18.063743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfe75-9082-4428-83c5-c04cc5d3f071 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:18.098440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:18.099319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:18.099604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:18.099908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] PATH_INFO: `/v3/users/5eeaca1f929d483b868965b6adddc8fd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:18.101163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] RBAC: Authorizing `identity:get_user(user_id=5eeaca1f929d483b868965b6adddc8fd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:18.102618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:18.104294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 567/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:17 2022] GET /identity/v3/users/5eeaca1f929d483b868965b6adddc8fd => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:18.110251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:18.199548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:18.215424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcfe30-ddcc-4f68-a0b8-745e176e2fac None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:18.253307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:18.254178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:18.254459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:18.254732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:18.256014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:18.257496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91903b-ffda-4271-8aaa-ff4a506c1084 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:18.259229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 555/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:18 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:18.265693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:18.295983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:18.374091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb06c98-8193-4456-bd52-a365c6a6f05e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:18.409703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:18.410590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:18.410871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:18.411148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef/users/5eeaca1f929d483b868965b6adddc8fd/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:18.413600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=369cf41efd1948228e4be5990028ecef, user_id=5eeaca1f929d483b868965b6adddc8fd, role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:18.415063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3e0b8-1918-42b7-be55-6f1f031c4ec9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:18.428994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 568/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:03:18 2022] PUT /identity/v3/projects/369cf41efd1948228e4be5990028ecef/users/5eeaca1f929d483b868965b6adddc8fd/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:03:20.414830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc1f12-cc27-427c-ad5e-3837302a6c77 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:20.415354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc1f12-cc27-427c-ad5e-3837302a6c77 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:20.415354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc1f12-cc27-427c-ad5e-3837302a6c77 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:20.416702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 556/1124] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:20.422013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:20.422431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:20.422546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:20.440272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:20.441444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:20.513839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 569/1125] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:20.537366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc1f12-cc27-427c-ad5e-3837302a6c77 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:20.643849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc1f12-cc27-427c-ad5e-3837302a6c77 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:20.678213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:20.679001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:20.679307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:20.679611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] PATH_INFO: `/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:20.681139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] RBAC: Authorizing `identity:get_role(role_id=6fdf8fef017f4d0d8d80f02776d28e74)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:20.682517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:20.684011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 557/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:20 2022] GET /identity/v3/roles/6fdf8fef017f4d0d8d80f02776d28e74 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:20.690226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:20.710566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:20.797109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06071206-dec8-40a0-88c1-334e7d59df97 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:20.832639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:20.833857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:20.834140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:20.834406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] PATH_INFO: `/v3/users/5eeaca1f929d483b868965b6adddc8fd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:20.835775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] RBAC: Authorizing `identity:get_user(user_id=5eeaca1f929d483b868965b6adddc8fd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:20.837210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:20.838845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 570/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:20 2022] GET /identity/v3/users/5eeaca1f929d483b868965b6adddc8fd => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:20.846668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:20.872997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:20.950867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59d26-3600-4980-b57e-b3b1a1327eec None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:20.988904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:20.989555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:20.989826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:20.990096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:21.058490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:21.059971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:21.082817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:21.170100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 558/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:21.176465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:21.264052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:21.280569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba456c23-124c-465a-aea7-f3b30a836d1e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:21.316642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:21.317505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:21.317785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:21.318062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] PATH_INFO: `/v3/projects/369cf41efd1948228e4be5990028ecef` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:21.319284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] RBAC: Authorizing `identity:get_project(project_id=369cf41efd1948228e4be5990028ecef)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:21.320787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eef618-2969-4fe8-8b47-05ff98f40e4d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:21.322495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 571/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:21 2022] GET /identity/v3/projects/369cf41efd1948228e4be5990028ecef => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:21.328966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:21.430674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce6818-00fb-4847-954d-062c5819eed4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:21.453900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:21.467820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:21.468484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:21.468764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:21.469035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:21.469881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] RBAC: Adding query filter params (role.id=6fdf8fef017f4d0d8d80f02776d28e74, scope.project.id=369cf41efd1948228e4be5990028ecef, user.id=5eeaca1f929d483b868965b6adddc8fd) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:21.470189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:21.471654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe00fe-d637-4af3-a8a8-0f0478693835 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:21.477906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 559/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:21 2022] GET /identity/v3/role_assignments?user.id=5eeaca1f929d483b868965b6adddc8fd&scope.project.id=369cf41efd1948228e4be5990028ecef&role.id=6fdf8fef017f4d0d8d80f02776d28e74 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:23.481878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f78d6-f948-4176-8a92-ae8e728acdb6 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:23.482421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f78d6-f948-4176-8a92-ae8e728acdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:23.482421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f78d6-f948-4176-8a92-ae8e728acdb6 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:23.483741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 572/1131] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:23.489063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:23.489517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:23.489626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:23.507650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:23.508856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:23.585839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 560/1132] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:23.609646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f78d6-f948-4176-8a92-ae8e728acdb6 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:23.717592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f78d6-f948-4176-8a92-ae8e728acdb6 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:23.753421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:23.754153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:23.754432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:23.754707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:23.755982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:23.757513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d294c6-e41a-4dd5-b7d2-ae987f9e5bce None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:23.759137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 573/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:23.765456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:23.872327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8896c34-b558-4d90-9205-3afb860bb507 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:23.901363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:23.925682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:23.926355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:23.926627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:23.926892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:23.927647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:23.929086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26037-2dcc-4d50-b651-42d12c2c3943 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:23.957806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 561/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:03:23 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:03:26.007608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078f74-8dc3-498b-b837-d87ff15c32f0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:26.008167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078f74-8dc3-498b-b837-d87ff15c32f0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:26.008167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74078f74-8dc3-498b-b837-d87ff15c32f0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:26.009481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 574/1135] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:26.015132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:26.015576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:26.015722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:26.033618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:26.034842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:26.105168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 562/1136] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:26.130016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74078f74-8dc3-498b-b837-d87ff15c32f0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:26.156396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:26.237075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74078f74-8dc3-498b-b837-d87ff15c32f0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:26.273204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:26.273900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:26.274177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:26.274450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:26.275685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:26.277131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2866ca4-7e8e-4218-beca-3bc1a0d5198f None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:26.278751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 575/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:03:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:26.285012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:26.381842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:26.385537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a466c-b81d-4256-b3cb-8ab101d51c37 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:26.423188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:26.423887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:26.424167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:26.424438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:26.425145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:26.426585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5b267-5475-45bd-96e8-42d009b81b90 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:26.446947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 563/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:03:26 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:03:28.494165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da37-7d77-4302-8d46-62be9e5001dc None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:28.494718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da37-7d77-4302-8d46-62be9e5001dc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:28.494718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1da37-7d77-4302-8d46-62be9e5001dc None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:28.496143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 576/1139] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:28.501581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:28.502018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:28.502123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:28.519513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:28.520771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:28.588954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 564/1140] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:28.612531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1da37-7d77-4302-8d46-62be9e5001dc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:28.712367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:28.723517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1da37-7d77-4302-8d46-62be9e5001dc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:28.757825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:28.758541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:28.758816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:28.759082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:28.770124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:28.777832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:28.778886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:28.779951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 577/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:28.785327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:28.870115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:28.888306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2431ad7-5f7e-4326-b268-a1b8d7125262 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:28.923754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:28.924429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:28.924709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:28.924984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:28.925771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:28.926072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:28.927456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:28.936431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 565/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:28.942623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:29.051440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aaaa9-55a3-4449-8fbd-753f15d890e0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:29.088001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:29.088885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:29.089163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:29.089435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] PATH_INFO: `/v3/users/acbe2797d872494a99eb895ba2d4ccef` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:29.103391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] RBAC: Authorizing `identity:get_user(user_id=acbe2797d872494a99eb895ba2d4ccef)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:29.104879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:29.106547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 578/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:28 2022] GET /identity/v3/users/acbe2797d872494a99eb895ba2d4ccef => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:29.112651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:29.218261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac826e1-1365-4b1f-a96e-d3844aeda25b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:29.253723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:29.254362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:29.254640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:29.254911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:29.269753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:29.324988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:29.326356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:29.426044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 566/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:29.432453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:29.443727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:29.534749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ce984-3a29-4b5d-a823-23a00a6c325b None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:29.569357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:29.570320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:29.570592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:29.570861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] PATH_INFO: `/v3/projects/bd526f0f89f64419b7bed61d5adf18f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:29.572106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] RBAC: Authorizing `identity:get_project(project_id=bd526f0f89f64419b7bed61d5adf18f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:29.573546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a8151-894b-4f8d-b21d-db0864a0348d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:29.575300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 579/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:29 2022] GET /identity/v3/projects/bd526f0f89f64419b7bed61d5adf18f6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:29.581753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:29.628923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:29.687306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353fb8a-d175-4162-be06-38d35a88c384 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:29.723519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:29.724206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:29.724484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:29.724714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:29.725577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=bd526f0f89f64419b7bed61d5adf18f6, user.id=acbe2797d872494a99eb895ba2d4ccef) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:29.725943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:29.727376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d04c1-f98e-4f80-aaba-30a4a28e094e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:29.732942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 567/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:29 2022] GET /identity/v3/role_assignments?user.id=acbe2797d872494a99eb895ba2d4ccef&scope.project.id=bd526f0f89f64419b7bed61d5adf18f6&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:31.716804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8ae8-22c0-4372-a9f4-b91c9b68b009 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:31.717363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8ae8-22c0-4372-a9f4-b91c9b68b009 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:31.717363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8ae8-22c0-4372-a9f4-b91c9b68b009 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:31.718724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 580/1147] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:31.724062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:31.724511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:31.724603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:31.741741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:31.742963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:31.809964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 568/1148] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:31.832629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a8ae8-22c0-4372-a9f4-b91c9b68b009 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:31.917261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:31.943907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a8ae8-22c0-4372-a9f4-b91c9b68b009 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:31.980599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:31.981335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:31.981616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:31.981891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:31.992511 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:32.000096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:32.001034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:32.001922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 581/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:32.007793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:32.050865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:32.108374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f83a07-4738-40d2-bc3c-75b48a32ae2e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:32.144012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:32.144718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:32.145005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:32.145265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:32.146030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:32.146338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:32.149311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:32.156654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 569/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:32.162866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:32.269122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483844c-4d61-48d6-a1c9-3302cf1c8aa6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:32.306322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:32.307212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:32.307530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:32.307805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] PATH_INFO: `/v3/users/acbe2797d872494a99eb895ba2d4ccef` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:32.309060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] RBAC: Authorizing `identity:get_user(user_id=acbe2797d872494a99eb895ba2d4ccef)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:32.310522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:32.312204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 582/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:32 2022] GET /identity/v3/users/acbe2797d872494a99eb895ba2d4ccef => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:32.318768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:32.423602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8b7aa-8a57-47a6-9c35-78b0cc30ef62 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:32.425154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:32.461418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:32.462405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:32.462685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:32.462962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] PATH_INFO: `/v3/projects/bd526f0f89f64419b7bed61d5adf18f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:32.464213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] RBAC: Authorizing `identity:get_project(project_id=bd526f0f89f64419b7bed61d5adf18f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:32.465690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d8745-f91d-4e2b-b7bf-8bc974eb01dc None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:32.467429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 570/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:32 2022] GET /identity/v3/projects/bd526f0f89f64419b7bed61d5adf18f6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:32.473965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:32.478187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:32.574689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70aff3-3de3-47e8-8a9a-ee312840cf12 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:32.609174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:32.610051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:32.610327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:32.610594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] PATH_INFO: `/v3/projects/bd526f0f89f64419b7bed61d5adf18f6/users/acbe2797d872494a99eb895ba2d4ccef/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:32.612642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] RBAC: Authorizing `identity:create_grant(project_id=bd526f0f89f64419b7bed61d5adf18f6, user_id=acbe2797d872494a99eb895ba2d4ccef, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:32.614105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b19b13-2b18-4dba-846f-fb0f76ed28bb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:32.628279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 583/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:03:32 2022] PUT /identity/v3/projects/bd526f0f89f64419b7bed61d5adf18f6/users/acbe2797d872494a99eb895ba2d4ccef/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:03:34.609885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbca53-0a96-4c95-90a5-56fbd370ea6e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:34.610437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbca53-0a96-4c95-90a5-56fbd370ea6e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:34.610437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbca53-0a96-4c95-90a5-56fbd370ea6e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:34.611761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 571/1154] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:34.617191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:34.617614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:34.617725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:34.635169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:34.636410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:34.698007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:34.707639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 584/1155] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:34.732189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbca53-0a96-4c95-90a5-56fbd370ea6e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:34.852983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbca53-0a96-4c95-90a5-56fbd370ea6e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:34.888283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:34.889001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:34.889284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:34.889558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:34.900175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:34.907715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:34.908765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:34.909776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 572/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:34.915515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:35.018305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24440b-1631-4250-b8f6-be6b471ef245 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:35.053996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:35.054740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:35.055008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:35.055282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:35.056063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:35.056369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:35.057755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:35.066263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 585/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:35.072443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:35.104602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:35.176806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec434b6-dc45-4a9b-8e82-a33ef942feab None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:35.211784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:35.212668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:35.212942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:35.213215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] PATH_INFO: `/v3/users/acbe2797d872494a99eb895ba2d4ccef` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:35.214461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] RBAC: Authorizing `identity:get_user(user_id=acbe2797d872494a99eb895ba2d4ccef)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:35.215950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:35.217663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 573/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:35 2022] GET /identity/v3/users/acbe2797d872494a99eb895ba2d4ccef => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:35.223958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:35.281856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:35.327217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50450618-3fa5-488b-9e94-7b4140645824 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:35.362063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:35.362705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:35.362980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:35.363250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:35.430913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:35.432311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:35.486848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:35.531473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 586/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:35.537808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:35.634621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:35.639232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574f438-4398-4e3f-af99-2e590379243e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:35.674419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:35.675270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:35.675571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:35.675847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] PATH_INFO: `/v3/projects/bd526f0f89f64419b7bed61d5adf18f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:35.677065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] RBAC: Authorizing `identity:get_project(project_id=bd526f0f89f64419b7bed61d5adf18f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:35.678504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d752bb5-71fb-4d20-a274-84864b144268 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:35.680269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 574/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:35 2022] GET /identity/v3/projects/bd526f0f89f64419b7bed61d5adf18f6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:35.686797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:35.792569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9bce9-5a94-4166-a3d0-256363334278 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:35.827860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:35.828515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:35.828801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:35.829064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:35.829900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=bd526f0f89f64419b7bed61d5adf18f6, user.id=acbe2797d872494a99eb895ba2d4ccef) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:35.830211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:35.831684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44c16e-1774-4bcc-8af7-19fad97aa607 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:35.838029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 587/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:35 2022] GET /identity/v3/role_assignments?user.id=acbe2797d872494a99eb895ba2d4ccef&scope.project.id=bd526f0f89f64419b7bed61d5adf18f6&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:37.865339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3691107-0306-4d3d-91f8-d987c5bd2e99 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:37.865876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3691107-0306-4d3d-91f8-d987c5bd2e99 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:37.865876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3691107-0306-4d3d-91f8-d987c5bd2e99 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:37.867289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 575/1162] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:37.873716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:37.874119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:37.874286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:37.892412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:37.893640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:37.908509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:37.966691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 588/1163] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:37.989757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3691107-0306-4d3d-91f8-d987c5bd2e99 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:38.095988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3691107-0306-4d3d-91f8-d987c5bd2e99 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:38.131646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:38.132368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:38.132646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:38.132917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:38.168660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:38.203924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:38.203924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:38.307162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 576/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:38.313534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:38.410387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df96c8-140e-4a41-b494-8c8babc253cb None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:38.444569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:38.445362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:38.445639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:38.445905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:38.452927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:38.454366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-47d52b77-b767-4cd4-82a3-0ed9660c6afe None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:38.469496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:38.471984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:38.473939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 589/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:03:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:38.477817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:38.555376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:38.592252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09d18-884d-43b7-954c-c3970f81afad None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:38.626577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:38.627230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:38.627519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:38.627802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:38.628534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:38.628833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:38.630206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb0fd-d9de-458c-bcee-34d51253edd6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:38.637413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 577/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Thu Jul 21 12:03:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:40.732993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac06ba6-dab9-4de4-ab3f-7372aac7d1cf None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:40.733556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac06ba6-dab9-4de4-ab3f-7372aac7d1cf None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:40.733556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac06ba6-dab9-4de4-ab3f-7372aac7d1cf None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:40.734931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 590/1167] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:40.740391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a7b5-5d6e-44a1-a642-b589c28268dc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:40.740816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a7b5-5d6e-44a1-a642-b589c28268dc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:40.740923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a7b5-5d6e-44a1-a642-b589c28268dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:40.758740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a038a7b5-5d6e-44a1-a642-b589c28268dc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:40.759978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a038a7b5-5d6e-44a1-a642-b589c28268dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:40.828642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 578/1168] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:40.852571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac06ba6-dab9-4de4-ab3f-7372aac7d1cf None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:40.865912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:40.960491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac06ba6-dab9-4de4-ab3f-7372aac7d1cf None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:40.994972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:40.995568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:40.995873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:40.996151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:40.996906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:40.998295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442176-73cf-4c9f-addc-f34b1f05c459 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:41.014207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 591/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Thu Jul 21 12:03:40 2022] POST /identity/v3/domains => generated 267 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:03:43.055899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56951dc-270b-41be-a547-732e782a0829 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:43.056497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56951dc-270b-41be-a547-732e782a0829 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:43.056497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56951dc-270b-41be-a547-732e782a0829 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:43.057806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 579/1170] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:43.063950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:43.064508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:43.064655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:43.082861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:43.084469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:43.149396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:43.155589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 592/1171] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:43.177369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56951dc-270b-41be-a547-732e782a0829 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:43.240417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:43.283997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56951dc-270b-41be-a547-732e782a0829 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:43.318507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:43.319373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:43.319681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:43.319957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] PATH_INFO: `/v3/domains/85af373755794c68ba9ccaf57af5cd17` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:43.330578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=85af373755794c68ba9ccaf57af5cd17)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:43.332080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f42175-3c8a-4555-adcf-5f9ede615280 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:43.334089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 580/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Jul 21 12:03:43 2022] GET /identity/v3/domains/85af373755794c68ba9ccaf57af5cd17 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:43.340343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:43.442278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90073df-a4d2-45aa-bd42-e544cadfdbdf None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:43.479758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:43.480422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:43.480702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:43.480959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:43.481673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:43.483137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f721bf-a214-46c3-a4c9-8688845217c7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:43.510677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 593/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:03:43 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:03:45.593391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036606a-dd15-4cfd-be40-49df6ca09497 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:45.593980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036606a-dd15-4cfd-be40-49df6ca09497 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:45.593980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036606a-dd15-4cfd-be40-49df6ca09497 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:45.595240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 581/1174] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:45.600655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:45.601099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:45.601199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:45.618495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:45.619727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:45.668239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:45.689658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 594/1175] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:45.715166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036606a-dd15-4cfd-be40-49df6ca09497 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:45.821128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036606a-dd15-4cfd-be40-49df6ca09497 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:45.841274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:45.856825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:45.857632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:45.857910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:45.858185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] PATH_INFO: `/v3/domains/85af373755794c68ba9ccaf57af5cd17` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:45.859402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] RBAC: Authorizing `identity:get_domain(domain_id=85af373755794c68ba9ccaf57af5cd17)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:45.860886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926bf8a-52b4-4fba-b42c-830d99d5292c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:45.862533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 582/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Thu Jul 21 12:03:45 2022] GET /identity/v3/domains/85af373755794c68ba9ccaf57af5cd17 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:45.868690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:45.970429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0fdea-faa3-46d6-82fc-5f29695a54b3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:46.005758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:46.006420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:46.006729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:46.007015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:46.007752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:46.009200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ec216-8786-4fcd-b286-a3b405f51c68 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:46.030395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 595/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:03:45 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:03:48.097567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4a48-6783-480f-a736-72696d014ce5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.098126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4a48-6783-480f-a736-72696d014ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.098126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4a48-6783-480f-a736-72696d014ce5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.099531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 583/1178] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:48.105252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.105690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.105786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.123898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:48.125108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:48.156692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:48.194314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 596/1179] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:48.218489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4a48-6783-480f-a736-72696d014ce5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:48.323451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4a48-6783-480f-a736-72696d014ce5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:48.358805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:48.359557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.359835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.360109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.370634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:48.378230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:48.379361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:48.380688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 584/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:48.385860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:48.493181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af4393-f5ee-4ff8-8c19-8410dc7fe30d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:48.528031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:48.528701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.528978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.529260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.529997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:48.530303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:48.531747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:48.540067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 597/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:03:48.546257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:48.598980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:48.655924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4be78-c158-4f5b-9758-7abbb22d3081 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:48.690587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:48.691464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.691775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.692051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] PATH_INFO: `/v3/users/bcdc6c910d844294a3255da80deee432` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.705913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] RBAC: Authorizing `identity:get_user(user_id=bcdc6c910d844294a3255da80deee432)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:48.707503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:48.709732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 585/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:48 2022] GET /identity/v3/users/bcdc6c910d844294a3255da80deee432 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:48.715823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:48.746337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:48.816913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853028c8-e6e8-4e72-b06e-f1fc89fe76bd None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:48.851826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:48.852465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:48.852750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:48.853023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:48.921047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:48.922423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:48.951910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:49.024062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 598/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:49.030289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:49.128414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdcd7e-1a25-4991-b5bf-b711d4fda18a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:49.136433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:49.175183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:49.176009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:49.176285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:49.176555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] PATH_INFO: `/v3/projects/1ecfb25866134a58b64575d26cf82c11` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:49.177789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] RBAC: Authorizing `identity:get_project(project_id=1ecfb25866134a58b64575d26cf82c11)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:49.179264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf30c7a-65cf-4b3b-8631-9fbbb644fb39 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:49.181007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 586/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:49 2022] GET /identity/v3/projects/1ecfb25866134a58b64575d26cf82c11 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:49.187588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:49.292661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa434-46fc-41da-8aae-12ebb40d16db None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:49.321826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:49.328390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:49.329045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:49.329328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:49.329599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:49.330433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=1ecfb25866134a58b64575d26cf82c11, user.id=bcdc6c910d844294a3255da80deee432) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:49.330739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:49.332212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0b41d-a425-4504-afbf-7aa83bdaa507 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:49.338362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 599/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:49 2022] GET /identity/v3/role_assignments?user.id=bcdc6c910d844294a3255da80deee432&scope.project.id=1ecfb25866134a58b64575d26cf82c11&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:51.359641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8f35-903c-4d7a-ac0f-8003ec8221c8 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:51.360235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8f35-903c-4d7a-ac0f-8003ec8221c8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:51.360235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8f35-903c-4d7a-ac0f-8003ec8221c8 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:51.361551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 587/1186] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:51.367310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:51.367804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:51.367906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:51.385142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:51.386345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:51.456108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 600/1187] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:03:51.480434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8f35-903c-4d7a-ac0f-8003ec8221c8 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:51.585914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8f35-903c-4d7a-ac0f-8003ec8221c8 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:51.620431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:51.621223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:51.621509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:51.621779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:51.632874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:51.640503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:51.641481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:03:51.642368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 588/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:51.648104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:51.750454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ff05e-a789-465a-b0f2-002ac7012009 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:51.774398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:51.786222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:51.786892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:51.787174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:51.787449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:51.788269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:51.788574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:51.789959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:51.798386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 601/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:51.805076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:51.843959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:51.911909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774ba7e-d2b6-40da-a4d3-f4a2fcb3ad0e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:51.946115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:51.946986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:51.947269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:51.947530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] PATH_INFO: `/v3/users/bcdc6c910d844294a3255da80deee432` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:51.948801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=bcdc6c910d844294a3255da80deee432)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:51.950241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:51.951935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 589/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:51 2022] GET /identity/v3/users/bcdc6c910d844294a3255da80deee432 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:51.958982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:52.060378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447151ac-b591-43f7-80f4-cc178b4cbfb7 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:52.095086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:52.095971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:52.096254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:52.096536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] PATH_INFO: `/v3/projects/1ecfb25866134a58b64575d26cf82c11` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:52.097768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] RBAC: Authorizing `identity:get_project(project_id=1ecfb25866134a58b64575d26cf82c11)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:52.099215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed47c4ed-4a55-4317-8f89-fc0c8a718d67 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:52.101017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 602/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:51 2022] GET /identity/v3/projects/1ecfb25866134a58b64575d26cf82c11 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:52.107536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:52.212989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43dbd-a72f-41f4-b659-7f32b38d3ba2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:52.214587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:52.248806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:52.249709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:52.249983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:52.250260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] PATH_INFO: `/v3/projects/1ecfb25866134a58b64575d26cf82c11/users/bcdc6c910d844294a3255da80deee432/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:52.252368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ecfb25866134a58b64575d26cf82c11, user_id=bcdc6c910d844294a3255da80deee432, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:52.253822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d53ff-073d-4dd7-acc1-c7f6ec82bd92 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:52.268610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 590/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:03:52 2022] PUT /identity/v3/projects/1ecfb25866134a58b64575d26cf82c11/users/bcdc6c910d844294a3255da80deee432/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:03:54.249789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918b6db-07dc-4161-b86f-4b9f57ad30de None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.250370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918b6db-07dc-4161-b86f-4b9f57ad30de None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.250370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918b6db-07dc-4161-b86f-4b9f57ad30de None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:54.251716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 603/1193] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:03:54.257456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dff23-0475-47d9-8816-08778496e9ba None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.257906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dff23-0475-47d9-8816-08778496e9ba None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.258007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dff23-0475-47d9-8816-08778496e9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:54.275949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-780dff23-0475-47d9-8816-08778496e9ba None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:03:54.276978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-780dff23-0475-47d9-8816-08778496e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:03:54.348968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 591/1194] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:03:54.372809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918b6db-07dc-4161-b86f-4b9f57ad30de None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:54.432771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:54.486571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918b6db-07dc-4161-b86f-4b9f57ad30de None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:54.522738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:54.523534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.523832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.524111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:54.535516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:03:54.543152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:03:54.544260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:03:54.545357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 604/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:03:54.550665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dff23-0475-47d9-8816-08778496e9ba None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:54.641931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:54.651831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dff23-0475-47d9-8816-08778496e9ba None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:54.687255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:54.687877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.688157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.688434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:54.689188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:54.689495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:54.690869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:54.699354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 592/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:54.705957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:54.811473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec053-0bbe-4434-9642-9bb9b8fd4f71 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:54.825561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:54.846183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:54.847034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.847304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.847601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] PATH_INFO: `/v3/users/bcdc6c910d844294a3255da80deee432` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:54.848821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] RBAC: Authorizing `identity:get_user(user_id=bcdc6c910d844294a3255da80deee432)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:54.850290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:54.852026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 605/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Thu Jul 21 12:03:54 2022] GET /identity/v3/users/bcdc6c910d844294a3255da80deee432 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:54.858064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:54.962567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1df6-2831-43bc-8f52-28962cc463f9 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:54.996487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:54.997133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:54.997384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:54.997689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:55.021559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:55.065022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:55.066506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:55.165812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 593/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:03:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:03:55.171941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:55.272720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf49f8-0be7-4c36-8857-9119c8c2b9ad None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:55.309572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:55.310437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:55.310719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:55.310996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] PATH_INFO: `/v3/projects/1ecfb25866134a58b64575d26cf82c11` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:55.312260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=1ecfb25866134a58b64575d26cf82c11)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:55.313714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d971f-cee8-4104-9ad0-124647199ca9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:55.315432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 606/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Thu Jul 21 12:03:55 2022] GET /identity/v3/projects/1ecfb25866134a58b64575d26cf82c11 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:03:55.321991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:03:55.380727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:03:55.427316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824af5-6b3e-403f-9677-259717d38abc None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:03:55.462773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:03:55.463432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:03:55.463751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:03:55.464027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:03:55.464874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=1ecfb25866134a58b64575d26cf82c11, user.id=bcdc6c910d844294a3255da80deee432) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:03:55.465200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:03:55.466652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f060ca-0ee4-410f-989e-7694ce5c65d4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:03:55.472896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 594/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:03:55 2022] GET /identity/v3/role_assignments?user.id=bcdc6c910d844294a3255da80deee432&scope.project.id=1ecfb25866134a58b64575d26cf82c11&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:04:05.304539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=75 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:05.470235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=41 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:54.115881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0e33-f989-4da7-b700-44c296ce2738 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:54.117116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0e33-f989-4da7-b700-44c296ce2738 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:54.117602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0e33-f989-4da7-b700-44c296ce2738 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:54.119510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 607/1201] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:04:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:04:54.125346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:54.125957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:54.125957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:54.162468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:04:54.164127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:04:54.284037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 595/1202] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:04:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:04:54.314130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0e33-f989-4da7-b700-44c296ce2738 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:54.493872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0e33-f989-4da7-b700-44c296ce2738 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:54.547538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:54.547538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:54.547538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:54.547538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:54.570469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:54.572317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:04:54.594961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:04:54.596119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:04:54.597126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:04:54.597126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:04:54.597126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:04:54.597126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:04:54.599748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 608/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:04:54.605564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:54.760407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2133f-d17e-4852-a1e2-57c366cfc951 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:54.802989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:54.814591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:54.815322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:54.815663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:54.815947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:54.816730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:04:54.817049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:54.818580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5d1ff-8dc7-4939-90e1-c57e6a695bac None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:54.834809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 596/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:04:54.844005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:54.982427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:55.077269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89fc8-b735-40c9-ae5b-d98995c6e909 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:55.142075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:55.143120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:55.143120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:55.143411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:55.144579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:55.145942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd80629-da38-493b-bba5-50d3ec950277 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:55.187558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 609/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Thu Jul 21 12:04:54 2022] POST /identity/v3/users => generated 251 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:04:57.958269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8936b3-a2e9-48e1-9616-337e5af8b0d5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:57.958836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8936b3-a2e9-48e1-9616-337e5af8b0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:57.959075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8936b3-a2e9-48e1-9616-337e5af8b0d5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:57.960824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 597/1206] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:04:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:04:57.967291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:57.968013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:57.968013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:57.989889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:04:57.991896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:04:58.091582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 610/1207] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:04:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:04:58.132862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8936b3-a2e9-48e1-9616-337e5af8b0d5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:58.330287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8936b3-a2e9-48e1-9616-337e5af8b0d5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:58.390676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:58.392199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:58.392585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:58.392944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:58.410618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:04:58.423812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:04:58.425013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:04:58.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:04:58.426672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 598/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:04:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:04:58.436174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:58.500762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:58.589990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440eda5-ea87-4ffd-8d91-87a384f55002 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:58.631158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:58.631935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:58.632223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:58.632488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:58.633243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:04:58.633659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:58.635024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:58.646039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 611/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:04:58.653106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:58.761260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:58.834970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd9b70-69ee-4381-a320-60b5abdae058 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:58.883711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:58.884284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:58.884649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:58.885017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:58.900414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:58.902287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:04:58.929908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:04:58.931172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:04:58.939387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:04:58.939387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:04:58.939387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:04:58.939387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:04:58.939387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 599/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:04:58.945732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:59.120510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:59.124697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac11b2-e7c9-4048-95da-bb0a15a98f04 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:59.172576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:59.173300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:59.173729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:59.174161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:59.175362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:04:59.175929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:59.177447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:59.206244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 612/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:04:59.213647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:59.390167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38549b-c1b9-4e14-abea-d7ca83d812e0 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:59.392557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:04:59.446197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:59.447249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:59.447622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:59.448057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:59.460225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:59.462265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:04:59.482243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:04:59.484255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:04:59.485961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 600/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:04:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:04:59.495514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:59.667427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f96d5-3f0a-45b9-bb2e-23d1d0558442 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:59.717506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:59.718823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:59.719209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:59.719602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:04:59.720516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:04:59.720854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:04:59.722488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:04:59.748100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 613/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:04:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:04:59.754771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:04:59.916543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5f0c-8814-45fa-893e-39afd4e5d95a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:04:59.973017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:04:59.973867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:04:59.974270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:04:59.974728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:00.025665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:00.083999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:00.085885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:00.258153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 601/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:04:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:05:00.259913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:00.283816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:00.398995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ea50e-275b-4b40-8ea7-65e8e56e33a9 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:00.442382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:00.443190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:00.443553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:00.443790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:00.455899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:00.457804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:00.477572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:00.478619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:00.479898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:00.479898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:00.479898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:00.479898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:00.483957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 614/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:00.485765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:00.602242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:00.656763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d7ca6-a0e3-4d98-b27e-06d33864231f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:00.721561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:00.723190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:00.723700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:00.724229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:00.727035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:00.727035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:00.729590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:00.755057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 602/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:00.761460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:00.868921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:00.908230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b1f3-e8a2-417e-8107-e2f6f25c0144 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:00.956938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:00.958059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:00.958452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:00.958987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:00.971864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:00.974544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:00.999217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:01.000520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:01.001845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 615/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:01.009373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:01.177460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15487-def1-466e-8ab4-5ffd1bc3151a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:01.203441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:01.218745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:01.219454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:01.219751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:01.220064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:01.220931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:01.221247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:01.223135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c4411-5347-4e13-af53-c663949aa23b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:01.240605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 603/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:01.248619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:01.392904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a389-575f-4e27-aaaa-eba09da56eaa None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:01.435619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:01.447490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:01.448401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:01.448783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:01.449152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:01.450109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=76fd650362ab4b4f959c08d46e1827ca) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:01.450504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:01.452462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539aab5-82a4-4944-8b56-bdf96301216d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:01.460926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 616/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:05:01 2022] GET /identity/v3/role_assignments?user.id=76fd650362ab4b4f959c08d46e1827ca&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:03.827936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e613ea1-329e-4c45-a0ac-3bd90984a04e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:03.828562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e613ea1-329e-4c45-a0ac-3bd90984a04e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:03.828562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e613ea1-329e-4c45-a0ac-3bd90984a04e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:03.830043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 604/1220] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:03.836688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:03.837244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:03.837244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:03.855575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:03.856790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:03.944789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 617/1221] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:03.975458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e613ea1-329e-4c45-a0ac-3bd90984a04e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:04.108677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e613ea1-329e-4c45-a0ac-3bd90984a04e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:04.170960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:04.171991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:04.172285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:04.172594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:04.186667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:05:04.196532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:04.198214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:04.199097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 605/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:04.207538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:04.347031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf136c-0c34-4bc8-a125-937f643b23b2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:04.377477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:04.393225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:04.393998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:04.394279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:04.394604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:04.395533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] RBAC: Adding query filter params (name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:04.395854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:04.397581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:04.411598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 618/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:05:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:04.417163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:04.480941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:04.556408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63372bcc-ae61-41b5-865e-b5f2ab630241 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:04.612171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:04.613957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:04.614565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:04.615114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:04.634738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:04.637389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:04.663820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:04.665180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:04.666155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:04.666155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:04.666155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:04.666155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:04.666155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 606/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:04.674031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:04.811562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02723c8-7633-4bfc-95c6-6ec26adf3855 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:04.867125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:04.868546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:04.868988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:04.869419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:04.870675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:04.871085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:04.873008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:04.890289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 619/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:04.902377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:05.014565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:05.023361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467d0c9-2556-4c93-89ef-9645d40495b3 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:05.067820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:05.068741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:05.069048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:05.069366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:05.079348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:05.081058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:05:05.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:05.101514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:05.102799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 607/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:05.110771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:05.174069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:05.254717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798d08e-7b2e-487f-a5dc-911dac6ed73a None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:05.296322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:05.297043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:05.297316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:05.297631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:05.298625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:05.298927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:05.300620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:05.322999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 620/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:05:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:05.329134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:05.463299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef23e-2cd2-4987-aff3-bdb6a7985157 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:05.506314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:05.507136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:05.507413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:05.507747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:05.525226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:05.527716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:05.558361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:05.561812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:05.562694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:05.562694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:05.562694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:05.562694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:05.562694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 608/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:05.567174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:05.713282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7a55-563e-47c4-bda1-f9b8048c4c70 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:05.718822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:05.773215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:05.773935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:05.774212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:05.774524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:05.775395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:05.775734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:05.777236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:05.792168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 621/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:05.798747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:05.810021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:05.925365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c34-c704-4be1-8f35-e2afa114e479 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:05.967081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:05.967984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:05.968284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:05.968573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:05.976540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:05.978002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:05.992786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:05.995303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:05.996296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 609/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:06.000633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:06.158997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309c9f-a419-48e5-b9d5-27018d4ac5eb None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:06.201099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:06.201816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:06.202124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:06.202445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:06.203209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:06.203550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:06.205023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14202b-e49f-4fc2-a4ae-26c660e553dc None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:06.220201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 622/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:06.226652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:06.306359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:06.350289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9820ed-3900-4156-b040-7a0bc30a17a6 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:06.402925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:06.403884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:06.404158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:06.404429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/76fd650362ab4b4f959c08d46e1827ca/roles/2de4317bf4e54a4abdb2ef4d2498df92` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:06.428238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=76fd650362ab4b4f959c08d46e1827ca, role_id=2de4317bf4e54a4abdb2ef4d2498df92)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:06.430840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21ce97-ca68-4dc1-a4aa-a910ef0c91fd None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:06.455726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 610/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:05:06 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/76fd650362ab4b4f959c08d46e1827ca/roles/2de4317bf4e54a4abdb2ef4d2498df92 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:05:08.955134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137684-276c-4c83-bc0a-7c1884805ed2 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:08.955718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137684-276c-4c83-bc0a-7c1884805ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:08.955718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce137684-276c-4c83-bc0a-7c1884805ed2 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:08.957500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 623/1233] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:08.963731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:08.964322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:08.964322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:08.985174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:08.986434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:09.077555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 611/1234] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:09.105049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce137684-276c-4c83-bc0a-7c1884805ed2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:09.138642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:09.254868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce137684-276c-4c83-bc0a-7c1884805ed2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:09.295149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:09.296010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:09.296369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:09.296638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:09.310328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:05:09.319762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:09.323894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:09.324903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 624/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Thu Jul 21 12:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:05:09.331264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:09.405556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:09.489913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0cda6-68a5-4468-ae1f-c78c17917d73 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:09.534165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:09.535077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:09.535518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:09.535916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:09.537040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:09.537618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:09.540150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:09.551708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 612/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Thu Jul 21 12:05:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:09.558674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:09.675646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:09.697764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58908a4e-9608-46ba-a6c7-11abf12b0131 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:09.741193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:09.742049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:09.742346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:09.742621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:09.757337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:09.759660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:09.781134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:09.782136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:09.783074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:09.783074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:09.783074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:09.783074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:09.783074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 625/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:05:09.789082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:09.914137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a039bd9-8c5f-4f73-9d6b-b12a0fb097ae None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:09.926766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:09.955745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:09.956831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:09.957183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:09.957552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:09.958317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:09.958762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:09.960359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:09.975362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 613/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:09.981572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:10.115781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f51adf-09a5-46f8-96fb-08112d07ca4e None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:10.164660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:10.165377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:10.165701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:10.166010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:10.169866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:10.177516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:10.179128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:05:10.196937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:10.199270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:10.201409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 626/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:10.205492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:10.351743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7f1c6-eb22-4fec-98e6-1346c05b4f92 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:10.394961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:10.395594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:10.395887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:10.396176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:10.397109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:10.397409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:10.398854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:10.430045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 614/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:05:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:10.431509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:10.588057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ea95-587d-498f-b676-38100b937cb3 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:10.633811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:10.634520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:10.634874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:10.635128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:10.713710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:10.722656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:10.724900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:10.871920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 627/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:05:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:05:10.881828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:10.917569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:11.024856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad02e0-081d-4682-8388-b0e55e813977 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:11.076235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:11.077074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:11.077369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:11.077611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:11.089880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:11.091370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:11.110575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:11.111789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:11.112951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:11.112951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:11.112951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:11.112951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:11.112951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 615/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:11.118085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:11.239953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:11.248283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b44ba0-694f-4b0d-aa23-acd2c866a048 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:11.290811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:11.291357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:11.291674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:11.291913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:11.292652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:11.293029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:11.294485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:11.307310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 628/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:05:11.313938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:11.434611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:11.465056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1e6a-2170-4a8f-999c-5b49ff11dc5a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:11.508368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:11.509279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:11.509481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:11.509824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:11.519192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:11.520841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:11.538113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:11.540012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:11.542153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 616/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:11.547500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:11.690044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6de681-c90b-4ba3-9c6c-c20b8da3cb51 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:11.751820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:11.752592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:11.752869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:11.753221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:11.754240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:11.754478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:11.756205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b34ae9-04c5-419c-8392-ac32164b5f68 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:11.781881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 629/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:11.790475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:11.954300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790cfa6-c435-458a-94b3-8e7409a3b201 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:11.995361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:11.996097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:11.996408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:11.996750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:11.997699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] RBAC: Adding query filter params (role.id=2de4317bf4e54a4abdb2ef4d2498df92, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=76fd650362ab4b4f959c08d46e1827ca) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:11.998042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:11.999856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530793-2e0e-471a-adff-d0347d11795b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:12.007611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 617/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:05:11 2022] GET /identity/v3/role_assignments?user.id=76fd650362ab4b4f959c08d46e1827ca&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=2de4317bf4e54a4abdb2ef4d2498df92 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:14.478457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d198a-9ff8-44ec-bcdf-a78515a29316 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:14.479158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d198a-9ff8-44ec-bcdf-a78515a29316 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:14.479158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d198a-9ff8-44ec-bcdf-a78515a29316 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:14.481037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 630/1247] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:05:14.487833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bed13-e9ba-409d-867b-928336161cbe None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:14.488382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bed13-e9ba-409d-867b-928336161cbe None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:14.488607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bed13-e9ba-409d-867b-928336161cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:14.494561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:14.513332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-534bed13-e9ba-409d-867b-928336161cbe None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:14.514657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-534bed13-e9ba-409d-867b-928336161cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:14.609180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 618/1248] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 12:05:14.653440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d198a-9ff8-44ec-bcdf-a78515a29316 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:14.680701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:14.813086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d198a-9ff8-44ec-bcdf-a78515a29316 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:14.870026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:14.870993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:14.871393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:14.871776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:14.885881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:05:14.894631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:14.899845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:14.900926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 631/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:05:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:14.906025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bed13-e9ba-409d-867b-928336161cbe None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:15.064393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bed13-e9ba-409d-867b-928336161cbe None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:15.111225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:15.118056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:15.118825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:15.119105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:15.119373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:15.120174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:15.120481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:15.121882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:15.139610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 619/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:05:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:15.141143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:15.281606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:15.316346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afd5fb-6122-4949-92a7-43ae263522ab None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:15.363515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:15.364685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:15.365022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:15.365341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:15.381418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:15.383243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:15.406451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:15.414357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:15.415463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:15.415463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:15.415463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:15.415463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:15.415463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 632/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:15.415838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:15.557947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5880909-6299-46ec-94df-ed4c863cf11e None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:15.602438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:15.603022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:15.603305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:15.603615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:15.604386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:15.604929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:15.606064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:15.619543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 620/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:15.626765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:15.766462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796eed42-71e4-4824-8bdf-f186ce47c91c None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:15.814529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:15.827176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:15.828805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:15.829337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:15.829763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:15.840409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:15.841921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:05:15.857962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:15.859138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:15.860444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 633/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:05:15.865641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:15.929808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:15.993874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97fca-3652-4686-887e-3de9be3e396a None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:16.043641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:16.046132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:16.046132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:16.046132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:16.047230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:16.047813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:16.050409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:16.077276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 621/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:05:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:16.085191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:16.234932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6402c-da49-4a1c-b6a6-d844d3591418 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:16.284976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:16.285865 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:16.286176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:16.286573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:16.365560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:16.379463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:16.383232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:16.528579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 634/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:05:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:05:16.530257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:16.660160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b62d4-ea4f-4120-ac70-af2aafcfa67c None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:16.676622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:16.707575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:16.708822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:16.709401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:16.709936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:16.732424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:16.735546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:16.769277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:16.770992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:16.772254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:16.772254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:16.772254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:16.772254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:16.772254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 622/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:16.777522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:16.896563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:16.913527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8312150-d091-401f-8341-132ee92c0c90 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:16.959211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:16.959986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:16.960434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:16.960861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:16.961705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:16.962256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:16.964637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:16.982619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 635/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:16.991759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:17.149320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20961-3adf-4dac-8519-9774cf88a5af None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:17.193799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:17.195017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:17.195407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:17.195877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:17.204665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:17.206536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:17.221772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:17.223052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:17.224112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 623/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:17.229850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:17.358000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda430d-105e-4906-bffe-cc2cd27873b5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:17.396516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:17.397305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:17.397684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:17.397959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:17.398764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:17.399075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:17.400725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d1c40-aed9-4d71-a225-16f47ad7b57d None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:17.420857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 636/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:17.430793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:17.470952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:17.563351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a93d5-6e39-4fe4-9a60-25641ba3b3b2 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:17.612834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:17.613500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:17.613785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:17.614067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:17.614974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=76fd650362ab4b4f959c08d46e1827ca) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:17.615288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:17.616827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d73378-ae8c-438f-aeb4-5ca6b2ad002f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:17.623863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 624/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:05:17 2022] GET /identity/v3/role_assignments?user.id=76fd650362ab4b4f959c08d46e1827ca&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:19.657596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e381b1-99b6-4637-aee2-7aee962952f1 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:19.658318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e381b1-99b6-4637-aee2-7aee962952f1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:19.658318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e381b1-99b6-4637-aee2-7aee962952f1 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:19.659908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 637/1261] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:19.665426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:19.665426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:19.666013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:19.683952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:19.685933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:19.763606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 625/1262] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:19.791700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e381b1-99b6-4637-aee2-7aee962952f1 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:19.803225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:19.915398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e381b1-99b6-4637-aee2-7aee962952f1 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:19.950029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:19.950828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:19.951107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:19.951373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:19.962336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:05:19.969946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:19.971147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:19.976628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 638/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:19.977725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:19.996809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:20.085042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe4ba7-87db-4de4-9682-0fa28eff67c9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:20.127371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:20.128185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:20.128471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:20.128874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:20.129840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:20.130169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:20.131617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:20.140414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 626/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:20.146660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:20.217477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:20.252952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc74-298a-402e-9964-0ae2204e50e5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:20.292533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:20.293302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:20.293577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:20.293854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:20.308642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:20.310149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:20.328011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:20.329029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:20.330108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:20.330108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:20.330108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:20.330108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:20.330108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 639/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:20.336087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:20.422193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:20.444598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a663e-0a25-4ea0-a21d-7f847f1c024f None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:20.482758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:20.483332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:20.483655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:20.483973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:20.484706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:20.485008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:20.486448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:20.498828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 627/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:20.505504 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:20.610670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deccad-0f59-45ab-b62d-eb8aae0660b6 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:20.631360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:20.646855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:20.647708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:20.647989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:20.648266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:20.656296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:20.657758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:05:20.675002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:20.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:20.677539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 640/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:20.683683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:20.794584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acb88a-f889-41d3-b05f-a19f26eb7a94 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:20.829087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:20.829754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:20.830035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:20.830302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:20.831523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:20.831834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:20.833281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:20.842625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:20.854457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 628/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:05:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:20.860429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:20.966610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a670d-5c11-4e33-98e3-afa568e904d0 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:21.001228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:21.002033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:21.002311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:21.002588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:21.015269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:21.016976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:21.035334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:21.036387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:21.037298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:21.037298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:21.037298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:21.037298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:21.037298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 641/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:21.042585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:21.146839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f595ef7-c499-483a-8298-2722344a09e4 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:21.186369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:21.187038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:21.187318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:21.187667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:21.188394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:21.188694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:21.190065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:21.201961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 629/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:05:21.208458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:21.243937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:21.314778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130471b-6c98-46f2-81c8-44bdc1a24bd8 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:21.350524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:21.351319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:21.351767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:21.352047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:21.359161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:21.360650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:21.374859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:21.375836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:21.376734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 642/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:21.382992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:21.451557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:21.499687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66af2c4-1006-4dc2-9ac2-b8bb2b579351 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:21.537563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:21.538295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:21.538575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:21.538856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:21.539761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:21.540075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:21.541517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22065c82-3f20-4625-91f9-d14215965f1b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:21.554087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 630/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:21.560684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:21.658341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:21.670466 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98e89-5b5d-4dae-8205-5067fecf45ef None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:21.708885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:21.709695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:21.709975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:21.710252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] PATH_INFO: `/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/76fd650362ab4b4f959c08d46e1827ca/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:21.712414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] RBAC: Authorizing `identity:create_grant(project_id=6a4db6abd1a244249ec2954699f21c5b, user_id=76fd650362ab4b4f959c08d46e1827ca, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:21.713909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778be53-7a0b-44a1-9d2a-70899cd006af None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:21.730354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 643/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Thu Jul 21 12:05:21 2022] PUT /identity/v3/projects/6a4db6abd1a244249ec2954699f21c5b/users/76fd650362ab4b4f959c08d46e1827ca/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:05:23.737786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e0bda-36d5-456f-a252-788e7bc85669 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:23.738418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e0bda-36d5-456f-a252-788e7bc85669 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:23.738418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e0bda-36d5-456f-a252-788e7bc85669 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:23.739833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 631/1274] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:23.745294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:23.745744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:23.745843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:23.764727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:23.766015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:23.836295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 644/1275] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:23.860757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e0bda-36d5-456f-a252-788e7bc85669 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:23.971748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e0bda-36d5-456f-a252-788e7bc85669 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:23.990642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:24.009411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.010189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.010471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.010750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.021973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 12:05:24.029484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:24.030622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:24.031692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 632/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Thu Jul 21 12:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 12:05:24.036902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:24.133675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:24.157245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486be8a-7ef7-414a-a7cb-34f01712e898 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:24.195304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.196009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.196292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.196566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.197315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:24.197620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:24.199006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:24.208053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 645/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:24.214665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:24.320337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdd458-8359-4341-81a7-38a1eb452b8b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:24.356308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.357107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.357386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.357666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.369517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:24.370985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:24.388929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:24.389903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:24.390826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:24.390826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:24.390826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:24.390826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:24.390826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 633/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:24.396624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:24.499370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1c84c-5580-491f-bfb9-d1504a6dd0b5 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:24.534136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.534805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.535085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.535357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.536119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:24.536483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:24.537867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:24.549789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 646/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:24.555780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:24.592464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:24.664537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5e0c0-dfb2-4591-a0bc-54dfe47abd74 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:24.699590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.700398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.700678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.700934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.709054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:24.710515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 12:05:24.725273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:24.726678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:24.727722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 634/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:24.734674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:24.741337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:24.859134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f9171-be13-4e25-b5de-3b5ed9cd1b00 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:24.895731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:24.896412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:24.896693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:24.896969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:24.897814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:24.898122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:24.899613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:24.916651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 647/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:24.923303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:25.026234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:25.036328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc9ad6-3d3f-4aac-8614-e09c4aedd686 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:25.071854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:25.072512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:25.072796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:25.073071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:25.146179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:25.147603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:25.242916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:25.254147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 635/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Thu Jul 21 12:05:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 12:05:25.260356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:25.327795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:25.367249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a6601-8189-4705-b44a-0955aa0b2087 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:25.402093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:25.402920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:25.403198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:25.403470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:25.414182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:25.415679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 12:05:25.433604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:25.434646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:25.435626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 12:05:25.435626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 12:05:25.435626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 12:05:25.435626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:25.435626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 648/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:25.442913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:25.555763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436cb5-21fd-4778-860d-1e1a3c59e115 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:25.592257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:25.592917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:25.593219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:25.593491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:25.594222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:25.594526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:25.596172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:25.609700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 636/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Thu Jul 21 12:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:25.616422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:25.727636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1e999-1d4a-46f1-b50b-2520f0bcfc94 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:25.742583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:25.766403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:25.767207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:25.767519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:25.767799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:25.775002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:25.776502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 12:05:25.790568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:25.791657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: ERROR keystone.server.flask.application Jul 21 12:05:25.792832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 649/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Thu Jul 21 12:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:25.798025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:25.852369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:25.907681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e96655-002e-417d-afef-7339a5c40989 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:25.941283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:25.941927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:25.942204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:25.942477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:25.943328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:25.943665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:25.945107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65e33b-3041-4b64-9166-bc12b0b08384 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:25.957617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 637/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Thu Jul 21 12:05:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:25.964091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:26.073687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795ccaa-8569-464f-b0a8-c3097ae772c1 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:26.108186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:26.108814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:26.109114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:26.109387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:26.110218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] RBAC: Adding query filter params (role.id=abe682bd8c304ac3993415ae5ef145ee, scope.project.id=6a4db6abd1a244249ec2954699f21c5b, user.id=76fd650362ab4b4f959c08d46e1827ca) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:26.110522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:26.112003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f310d6-1587-43ed-a604-768a35484c5e None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:26.118105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 650/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Thu Jul 21 12:05:25 2022] GET /identity/v3/role_assignments?user.id=76fd650362ab4b4f959c08d46e1827ca&scope.project.id=6a4db6abd1a244249ec2954699f21c5b&role.id=abe682bd8c304ac3993415ae5ef145ee => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:28.197120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb75f3a-8c5f-4cad-ae5e-bbf054755cbe None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:28.197692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb75f3a-8c5f-4cad-ae5e-bbf054755cbe None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:28.197692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb75f3a-8c5f-4cad-ae5e-bbf054755cbe None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:28.199127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 638/1288] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:28.204769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:28.204769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:28.205295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:28.222770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:28.224135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:28.254691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:28.308089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 651/1289] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:28.331340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb75f3a-8c5f-4cad-ae5e-bbf054755cbe None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:28.437533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb75f3a-8c5f-4cad-ae5e-bbf054755cbe None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:28.448475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:28.475349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:28.476311 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:28.476602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:28.477331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:28.478199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:28.479753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:28.492209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:28.493379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:28.494504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:28.498592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 639/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:05:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:28.499697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:28.617705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099de83-6e91-4cbc-a84b-9c6642f9a9b7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:28.652571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:28.653245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:28.653532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:28.653806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:28.654525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:28.654829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:28.656263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed382034-de63-443c-8574-3ec5b06cdc52 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:28.661619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 652/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Jul 21 12:05:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:28.667903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:28.786142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a18bb-7056-4184-82d9-dd19bcc4f58b None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:28.827357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:28.828066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:28.828370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:28.828673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] PATH_INFO: `/v3/services` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:28.829407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] RBAC: Adding query filter params (type=placement) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:28.829736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:28.831564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a870-ffb7-4485-b65c-30b3839176ad None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:28.836779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 640/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Jul 21 12:05:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:30.906556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a26d-9435-42d7-bdb9-1da80c9ac77e None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:30.907119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a26d-9435-42d7-bdb9-1da80c9ac77e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:30.907119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9a26d-9435-42d7-bdb9-1da80c9ac77e None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:30.908742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 653/1293] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:30.915310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ddadf-bcc1-4d37-be61-4ad8f14972be None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:30.916205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ddadf-bcc1-4d37-be61-4ad8f14972be None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:30.916369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ddadf-bcc1-4d37-be61-4ad8f14972be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:30.942132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9b3ddadf-bcc1-4d37-be61-4ad8f14972be None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:30.943514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9b3ddadf-bcc1-4d37-be61-4ad8f14972be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:30.982299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:31.028673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 641/1294] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:31.058994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9a26d-9435-42d7-bdb9-1da80c9ac77e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:31.097245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:31.173556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9a26d-9435-42d7-bdb9-1da80c9ac77e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:31.209730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:31.210421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:31.210706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:31.210981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:31.211799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:31.213233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126f5ae-88de-4c30-aad3-d88d1d0d489b None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:31.233412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 654/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Thu Jul 21 12:05:31 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:05:33.262956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856bd4-06ea-43d1-bf82-130d9ceee8f1 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:33.263580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856bd4-06ea-43d1-bf82-130d9ceee8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:33.263580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64856bd4-06ea-43d1-bf82-130d9ceee8f1 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:33.264912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 642/1296] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:33.270275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:33.270812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:33.270924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:33.288886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:33.290087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:33.371542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 655/1297] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:33.395912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64856bd4-06ea-43d1-bf82-130d9ceee8f1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:33.507471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64856bd4-06ea-43d1-bf82-130d9ceee8f1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:33.543953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:33.544756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:33.545036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:33.545310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:33.545972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:33.547378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:33.559423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:33.560667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:33.561513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 643/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:05:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:33.567314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:33.590355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:33.693223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cacf1-f0cf-4716-b8b5-ee35969152d3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:33.728005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:33.728671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:33.728945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:33.729214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:33.729923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:33.730223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:33.731645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:33.737037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 656/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Jul 21 12:05:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:33.743980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:33.776366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:33.855264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606010-b3f0-4f81-838b-104fcce9b471 None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:33.893827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:33.894586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:33.894914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:33.895349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:33.896281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] RBAC: Adding query filter params (interface=public, service_id=f00fb773780f4862ad9f01bd8e7e6037, region_id=RegionOne) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:33.896590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:33.897979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd0f2-5444-4962-a338-663ce0cf1dd2 None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:33.903456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 644/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Thu Jul 21 12:05:33 2022] GET /identity/v3/endpoints?service_id=f00fb773780f4862ad9f01bd8e7e6037&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:05:33.909500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:34.003234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:34.024763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057fafc-6a11-4f1c-88af-9745ecd78a23 None admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:34.068895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:34.069544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:34.069826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:34.070135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:34.070859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:34.072497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10a287-c455-4a44-a6e4-4e4791225726 None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:34.080696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 657/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Thu Jul 21 12:05:33 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:05:36.107203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd1f-78fd-47d0-be59-40b6df773dce None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:36.107861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd1f-78fd-47d0-be59-40b6df773dce None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:36.107861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9cd1f-78fd-47d0-be59-40b6df773dce None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:36.109180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 645/1302] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:05:36.114627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:36.115070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:36.115158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:36.132886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:05:36.134103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:05:36.218278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 658/1303] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:05:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 12:05:36.242549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cd1f-78fd-47d0-be59-40b6df773dce None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:36.339698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:36.347434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9cd1f-78fd-47d0-be59-40b6df773dce None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:36.384049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:36.384768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:36.385048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:36.385322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:36.385981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:36.387361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 12:05:36.399793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 12:05:36.400961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application with Lock( Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self._enter() Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application return self.creator() Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application created_value = creator( Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: ERROR keystone.server.flask.application Jul 21 12:05:36.401982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 646/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Thu Jul 21 12:05:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 12:05:36.407318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:36.488265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:36.514083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab55b53-78a6-4a8a-a2ad-88a3fc051bcd None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:36.550428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:36.551085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:36.551382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:36.551726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] PATH_INFO: `/v3/services` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:36.552445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:05:36.552749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:36.554120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536dc9c3-103b-48c4-ad39-530e03fcbafd None admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:36.559737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 659/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Thu Jul 21 12:05:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:05:36.565894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:05:36.669313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceda1d4-2a29-461e-8c06-9cfccd24255d None admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:05:36.703655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:05:36.704321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:05:36.704600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:05:36.704873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:05:36.705606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:05:36.707348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9dc7-407b-4880-8a26-1e1722daba4a None admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:05:36.726919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 647/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Thu Jul 21 12:05:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:05:46.556886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=30 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:05:46.722462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:45.564436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ae2-0568-40d9-8dbc-b9b76e517f12 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:45.565136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ae2-0568-40d9-8dbc-b9b76e517f12 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:45.565136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36990ae2-0568-40d9-8dbc-b9b76e517f12 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:45.567430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 660/1307] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:45.576259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac038b-c537-4d11-94c3-8c8060d6ee67 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:45.576803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac038b-c537-4d11-94c3-8c8060d6ee67 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:45.577127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac038b-c537-4d11-94c3-8c8060d6ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:45.760807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-50ac038b-c537-4d11-94c3-8c8060d6ee67 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:45.763814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-50ac038b-c537-4d11-94c3-8c8060d6ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:45.990760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 648/1308] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:46.043890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:46.044219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:46.044592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:46.046195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 661/1309] 10.222.0.18 () {54 vars in 965 bytes} [Thu Jul 21 12:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:46.054134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa534f1b-9d1b-422b-8611-0cbd59672c9f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:46.054576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa534f1b-9d1b-422b-8611-0cbd59672c9f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:46.054948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa534f1b-9d1b-422b-8611-0cbd59672c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:46.166171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-aa534f1b-9d1b-422b-8611-0cbd59672c9f None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:46.167654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-aa534f1b-9d1b-422b-8611-0cbd59672c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:46.194113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:46.265948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 649/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Jul 21 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:46.280366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:06:46.283344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:46.411416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b51cce-ee18-4cfe-bcb5-a0ad836dc70c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:06:46.501574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:06:46.502237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:46.502534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:46.502782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:46.504542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:46.570726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:46.643236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:06:46.644824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c8361-2412-4b70-88db-c2befd420468 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:06:46.669523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 662/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:06:49.373229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f3f6-6337-4246-8041-2d4ed0d407f9 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:49.373840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f3f6-6337-4246-8041-2d4ed0d407f9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:49.374032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f3f6-6337-4246-8041-2d4ed0d407f9 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:49.375784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 650/1312] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:49.382123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f53d-e8eb-467a-83b0-e0deb3df9b82 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:49.382768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f53d-e8eb-467a-83b0-e0deb3df9b82 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:49.382768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f53d-e8eb-467a-83b0-e0deb3df9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:49.463156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1a36f53d-e8eb-467a-83b0-e0deb3df9b82 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:49.464305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1a36f53d-e8eb-467a-83b0-e0deb3df9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:49.573678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 663/1313] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:49.629296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:49.629581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:49.629864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:49.631271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 651/1314] 10.222.0.18 () {54 vars in 965 bytes} [Thu Jul 21 12:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:49.637883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a91-6ee5-427d-bba4-f226a1797de5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:49.638176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a91-6ee5-427d-bba4-f226a1797de5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:49.638430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9a91-6ee5-427d-bba4-f226a1797de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:49.649614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:49.732370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3b6e9a91-6ee5-427d-bba4-f226a1797de5 None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:49.733529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3b6e9a91-6ee5-427d-bba4-f226a1797de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:49.848640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 664/1315] 10.222.0.18 () {58 vars in 1062 bytes} [Thu Jul 21 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:06:49.863463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:06:49.865915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:49.975586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6741a2-be52-4819-a217-3296c7532669 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:06:50.053371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:50.056398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:06:50.057240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:50.057761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:50.058176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:50.060078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:50.175085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:06:50.176617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f4492-f41d-48a7-a079-dc06ebf540c6 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:06:50.197773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 652/1316] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:06:52.774156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:52.774719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:52.774719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:52.776134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 665/1317] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:52.781629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efff14-b8d8-48de-9c9b-350f939f18b8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:52.782091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efff14-b8d8-48de-9c9b-350f939f18b8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:52.782197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efff14-b8d8-48de-9c9b-350f939f18b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:52.848782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f0efff14-b8d8-48de-9c9b-350f939f18b8 None None] MFA Rules not processed for user `a08f17ad10864a879676643d5994d963`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:52.850078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f0efff14-b8d8-48de-9c9b-350f939f18b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:52.904186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:52.949826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 653/1318] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:52.992667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:06:52.994238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:53.108579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f77f8e-39ae-4589-9b60-2db596927f5d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:06:53.111390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:53.189011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:06:53.189679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:53.189959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:53.190241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:53.191958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:53.310014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:06:53.311437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae800fdb-89cf-499a-bc94-c36df5d996db service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:06:53.332502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 666/1319] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:06:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:06:55.928905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:55.929522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:55.929522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:55.930838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 654/1320] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:55.936394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de6849-8574-46fb-b1d5-26afda15d537 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:55.936842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de6849-8574-46fb-b1d5-26afda15d537 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:55.936946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de6849-8574-46fb-b1d5-26afda15d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:55.979305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:56.006642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-22de6849-8574-46fb-b1d5-26afda15d537 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:56.007916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-22de6849-8574-46fb-b1d5-26afda15d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:56.110985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 667/1321] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:56.152184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:06:56.153751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:56.263523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02608f6d-0478-4b0d-a32c-d6f8802a8fd9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:06:56.345316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:06:56.345976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:56.346232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:56.346563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:56.348306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:56.356824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:56.463522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:06:56.465032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609787a-920a-4969-bf1a-851a8412d401 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:06:56.484620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 655/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:06:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:06:59.339547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:59.340131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:59.340131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:59.341486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 668/1323] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:06:59.346944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7806df-481f-428a-94d6-0ba62f2817b5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:59.347413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7806df-481f-428a-94d6-0ba62f2817b5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:59.347588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7806df-481f-428a-94d6-0ba62f2817b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:59.420374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-dc7806df-481f-428a-94d6-0ba62f2817b5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:06:59.421529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dc7806df-481f-428a-94d6-0ba62f2817b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:59.485457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:59.530527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 656/1324] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:06:59.567814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:06:59.569348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:59.694678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d1c84-c77d-42f3-bb74-6b61b8d87641 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:06:59.709620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:06:59.795946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:06:59.796621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:06:59.796895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:06:59.797165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:06:59.799070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:06:59.933775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:06:59.935241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617bc4-5127-4294-b923-9a8f39df0daa service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:06:59.956203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 669/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:02.079506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:02.080090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:02.080090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:02.082314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 657/1326] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:07:02.088906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3095-b53c-45e6-b46d-650c5604daf7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:02.089531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3095-b53c-45e6-b46d-650c5604daf7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:02.089531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3095-b53c-45e6-b46d-650c5604daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:02.143040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:02.164178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-156d3095-b53c-45e6-b46d-650c5604daf7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:02.165399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-156d3095-b53c-45e6-b46d-650c5604daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:02.268639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 670/1327] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:02.305131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:02.306678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:02.426732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a6376-f867-4b9b-a139-1dc669afb59c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:02.512959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:02.513624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:02.513929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:02.514179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:02.515925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:02.534973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:02.642550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:02.644053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242e72b-d015-4027-8834-b1b94e99af1d service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:02.664365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 658/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:04.767802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:04.768413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:04.768413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:04.769758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 671/1329] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:04.775348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9f0a-0c5a-43c8-b709-91385fe3149d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:04.775922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9f0a-0c5a-43c8-b709-91385fe3149d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:04.775922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9f0a-0c5a-43c8-b709-91385fe3149d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:04.847165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a91c9f0a-0c5a-43c8-b709-91385fe3149d None None] MFA Rules not processed for user `a08f17ad10864a879676643d5994d963`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:04.848422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a91c9f0a-0c5a-43c8-b709-91385fe3149d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:04.920837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:04.957655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 659/1330] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:04.996214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:04.997784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:05.117667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a74afa-1954-45b9-88bd-491f6df44e6f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:05.143533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:05.218800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:05.219496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:05.219785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:05.220053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:05.221877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:05.342034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:05.343462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229eb0a1-c479-4b9b-ab12-1dbdd3fec4c7 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:05.362842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 672/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:07.686231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:07.686800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:07.686800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:07.688187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 660/1332] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:07.693540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162564c3-b8dc-4924-b545-4038a2d1f6ee None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:07.693961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162564c3-b8dc-4924-b545-4038a2d1f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:07.694089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162564c3-b8dc-4924-b545-4038a2d1f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:07.762312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:07.765622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-162564c3-b8dc-4924-b545-4038a2d1f6ee None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:07.768049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-162564c3-b8dc-4924-b545-4038a2d1f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:07.872175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 673/1333] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:07.908483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:07.910073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:08.017951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984858cf-1d0e-4b78-a0df-55a7d55e4fd5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:08.094014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:08.094670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:08.094946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:08.095214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:08.096889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:08.123535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:08.218469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:08.219982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eedeb7-1f94-4a94-ae92-62516be2ccf1 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:08.239385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 661/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:10.332561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:10.333123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:10.333123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:10.334492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 674/1335] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:10.340018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70d3a-ff3a-4778-894f-bce75f3b5093 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:10.340468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70d3a-ff3a-4778-894f-bce75f3b5093 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:10.340563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd70d3a-ff3a-4778-894f-bce75f3b5093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:10.408735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0dd70d3a-ff3a-4778-894f-bce75f3b5093 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:10.409875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0dd70d3a-ff3a-4778-894f-bce75f3b5093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:10.491355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:10.514984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 662/1336] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:10.555945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:10.557529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:10.668033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbaee2-07cb-4b87-9f1c-35a85d61c8a9 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:10.697251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:10.744005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:10.744648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:10.744919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:10.745189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:10.746841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:10.862972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:10.864450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da44fe-8b0a-4bfa-a9ce-22f38a7eb686 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:10.883311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 675/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:13.321759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:13.322280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:13.322280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:13.323696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 663/1338] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:13.338771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81cd9-d2f5-433f-9e0a-42f37f443516 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:13.339353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81cd9-d2f5-433f-9e0a-42f37f443516 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:13.339353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af81cd9-d2f5-433f-9e0a-42f37f443516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:13.409604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7af81cd9-d2f5-433f-9e0a-42f37f443516 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:13.410825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7af81cd9-d2f5-433f-9e0a-42f37f443516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:13.419397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:13.511500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 676/1339] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:13.548819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:13.550410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:13.660047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702167a-c058-42bb-b527-a3533e6647d8 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:13.737108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:13.737956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:13.738370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:13.738785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:13.741000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:13.781250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:13.869634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:13.871097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff050d-190e-4f9c-864e-151695bf5ad8 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:13.890058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 664/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:17.988910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:17.989559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:17.989889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:17.991550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 677/1341] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:17.997725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6033e-f236-4114-9057-c18d296e94d0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:17.998633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6033e-f236-4114-9057-c18d296e94d0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:17.999061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6033e-f236-4114-9057-c18d296e94d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:18.079749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-57c6033e-f236-4114-9057-c18d296e94d0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:18.081244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-57c6033e-f236-4114-9057-c18d296e94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:18.194864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:18.213792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 665/1342] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:18.268223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:18.269938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:18.388829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b24d4e-3674-475e-8b70-9a34b1a5361f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:18.436431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:18.482363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:18.483184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:18.483558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:18.483892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:18.485802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:18.636359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:18.638111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b82456-6d23-4895-9f1c-8f0c3efd9d0d service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:18.659643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 678/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:21.799551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:21.800188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:21.800188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:21.801517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 666/1344] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:21.807397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317382bf-fab4-4a0b-9719-c2a25804ef27 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:21.807941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317382bf-fab4-4a0b-9719-c2a25804ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:21.808056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317382bf-fab4-4a0b-9719-c2a25804ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:21.878173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-317382bf-fab4-4a0b-9719-c2a25804ef27 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:21.879407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-317382bf-fab4-4a0b-9719-c2a25804ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:21.899173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:21.982967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 679/1345] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:22.021262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:22.022865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:22.136019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b6605-e60f-4fa3-a201-0bc3ed5ffe43 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:22.216328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:22.217001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:22.217279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:22.217559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:22.219394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:22.274974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:22.359034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:22.360571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42682a-b493-46a5-8a9f-64e485a4e89d service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:22.381837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 667/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:26.758110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:26.758698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:26.758912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:26.760624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 680/1347] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:07:26.768338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb85f-6214-49cf-80f6-2cbf4c9b8086 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:26.768905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb85f-6214-49cf-80f6-2cbf4c9b8086 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:26.768905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb85f-6214-49cf-80f6-2cbf4c9b8086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:26.846676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a82bb85f-6214-49cf-80f6-2cbf4c9b8086 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:26.847949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a82bb85f-6214-49cf-80f6-2cbf4c9b8086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:26.977461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:26.981193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 668/1348] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:27.018868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:27.020637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:27.169448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e1057-729f-4962-a865-fb1771041047 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:27.237461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:27.274312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:27.275085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:27.275444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:27.275776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:27.277519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:27.418210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:27.419709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43857-d293-422b-a689-90c89c56a82c service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:27.439717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 681/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:32.762772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cfb2-5552-4256-9a05-e32b98360b35 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:32.764718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cfb2-5552-4256-9a05-e32b98360b35 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:32.765459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895cfb2-5552-4256-9a05-e32b98360b35 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:32.779622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 669/1350] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:07:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:32.780694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce2f2-3142-4366-a9fc-3a76c4f4aee2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:32.780694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce2f2-3142-4366-a9fc-3a76c4f4aee2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:32.780694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ce2f2-3142-4366-a9fc-3a76c4f4aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:32.911100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-274ce2f2-3142-4366-a9fc-3a76c4f4aee2 None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:32.913405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-274ce2f2-3142-4366-a9fc-3a76c4f4aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:32.954334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:33.165470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 682/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:07:38.063917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a81f-e991-4e47-8ce3-50c987cbdbe2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:38.064716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a81f-e991-4e47-8ce3-50c987cbdbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:38.064889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650a81f-e991-4e47-8ce3-50c987cbdbe2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:38.068089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 670/1352] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:07:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:38.076755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0492-1bbc-445c-a024-18ee261e910e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:38.078168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0492-1bbc-445c-a024-18ee261e910e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:38.079020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b0492-1bbc-445c-a024-18ee261e910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:38.227358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-040b0492-1bbc-445c-a024-18ee261e910e None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:38.229395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-040b0492-1bbc-445c-a024-18ee261e910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:38.370337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:38.451137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 683/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:40.214458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec70db-3274-442b-b7aa-d2e7f7f732b0 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:40.214993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec70db-3274-442b-b7aa-d2e7f7f732b0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:40.214993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec70db-3274-442b-b7aa-d2e7f7f732b0 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:40.216706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 671/1354] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:40.223519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf508b2-d97f-4ee7-aada-105eea6f475f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:40.224030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf508b2-d97f-4ee7-aada-105eea6f475f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:40.224661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf508b2-d97f-4ee7-aada-105eea6f475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:40.337683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-dbf508b2-d97f-4ee7-aada-105eea6f475f None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:40.339156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dbf508b2-d97f-4ee7-aada-105eea6f475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:40.470897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 684/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:44.029091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882b0b6-d1b7-406f-86b3-3004a089c25f None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:44.029689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882b0b6-d1b7-406f-86b3-3004a089c25f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:44.029689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882b0b6-d1b7-406f-86b3-3004a089c25f None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:44.031072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 672/1356] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:44.036996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1c876-5efd-4bcc-92ff-a2d1e93708e1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:44.037502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1c876-5efd-4bcc-92ff-a2d1e93708e1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:44.037609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1c876-5efd-4bcc-92ff-a2d1e93708e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:44.084228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:44.140446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-94c1c876-5efd-4bcc-92ff-a2d1e93708e1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:44.142914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-94c1c876-5efd-4bcc-92ff-a2d1e93708e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:44.311110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 685/1357] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:44.367896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:44.368685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:44.369322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:44.371506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 673/1358] 10.222.0.18 () {54 vars in 962 bytes} [Thu Jul 21 12:07:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:44.378784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba153f-6077-4c22-b20b-be993a96d82d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:44.379419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba153f-6077-4c22-b20b-be993a96d82d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:44.380005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba153f-6077-4c22-b20b-be993a96d82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:44.489068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-beba153f-6077-4c22-b20b-be993a96d82d None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:44.490503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-beba153f-6077-4c22-b20b-be993a96d82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:44.560073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:44.692274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 686/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Jul 21 12:07:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:44.701469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:44.704856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:44.930165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff95c-e2b4-47a9-9581-706da73902a3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:45.091223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:45.153926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:45.154902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:45.155697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:45.156152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:45.158355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:45.281073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:45.282931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50676e0b-0966-4635-9ac1-a2776ad3bbb4 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:45.318021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 674/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:47.696135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:47.696762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:47.696762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:47.698280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 687/1361] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:47.705053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8ea78-c284-4775-be6b-087c81ba601b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:47.705600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8ea78-c284-4775-be6b-087c81ba601b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:47.705600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8ea78-c284-4775-be6b-087c81ba601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:47.785125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f7d8ea78-c284-4775-be6b-087c81ba601b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:47.786415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f7d8ea78-c284-4775-be6b-087c81ba601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:47.849741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:47.954580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 675/1362] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:48.016333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:48.018387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:48.239195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684129-b4a2-4292-9a81-4e8deef9207b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:48.249661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:48.377908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:48.378573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:48.378853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:48.379224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:48.380996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:48.491765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:48.493304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c277-52b0-4a57-9c75-4455607663bf service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:48.514081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 688/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:51.092071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa2c0-88d5-4321-8643-14b48589d47c None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:51.092654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa2c0-88d5-4321-8643-14b48589d47c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:51.092856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa2c0-88d5-4321-8643-14b48589d47c None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:51.095284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 676/1364] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:51.101750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9bc9b-c0da-4e56-8484-1d301acf55cd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:51.102366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9bc9b-c0da-4e56-8484-1d301acf55cd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:51.102366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9bc9b-c0da-4e56-8484-1d301acf55cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:51.118704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:51.188540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8ae9bc9b-c0da-4e56-8484-1d301acf55cd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:51.189785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8ae9bc9b-c0da-4e56-8484-1d301acf55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:51.315550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 689/1365] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:51.365355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:51.365668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:51.365933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:51.367826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 677/1366] 10.222.0.18 () {54 vars in 962 bytes} [Thu Jul 21 12:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:51.377788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5ec5-71ce-4c69-b0e6-71855224355a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:51.378173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5ec5-71ce-4c69-b0e6-71855224355a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:51.378901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f5ec5-71ce-4c69-b0e6-71855224355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:51.508912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5d7f5ec5-71ce-4c69-b0e6-71855224355a None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:51.510712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5d7f5ec5-71ce-4c69-b0e6-71855224355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:51.566252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:51.700375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 690/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Jul 21 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:07:51.711258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:51.714321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:51.954868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ca35c-013e-4c7e-ae71-82cf15f13272 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:51.981847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:52.117225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:52.119631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:52.121843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:52.122604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:52.126061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:52.300586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:52.302417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514eb38-e425-4712-8b4a-66af387580af service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:52.327656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 678/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:53.091320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861129e-68a0-4cff-be7f-d1400e6cf180 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:53.092196 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861129e-68a0-4cff-be7f-d1400e6cf180 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:53.092384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861129e-68a0-4cff-be7f-d1400e6cf180 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:53.094771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 691/1369] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:53.102852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0fab-96eb-4a0f-b911-ff91e4f95ba8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:53.103335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0fab-96eb-4a0f-b911-ff91e4f95ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:53.103733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0fab-96eb-4a0f-b911-ff91e4f95ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:53.159068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:53.187706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7e7e0fab-96eb-4a0f-b911-ff91e4f95ba8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:53.188884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7e7e0fab-96eb-4a0f-b911-ff91e4f95ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:53.337596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 679/1370] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:07:53.694592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:53.694900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:53.695246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:53.698460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 692/1371] 10.222.0.18 () {54 vars in 964 bytes} [Thu Jul 21 12:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:07:53.704140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b42a35-9e3a-47e6-9543-effa89bbb6d4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:53.704432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b42a35-9e3a-47e6-9543-effa89bbb6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:53.704839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b42a35-9e3a-47e6-9543-effa89bbb6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:53.794728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-31b42a35-9e3a-47e6-9543-effa89bbb6d4 None None] MFA Rules not processed for user `5493d9cde9404225bedc423f0ecf213c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:53.796176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-31b42a35-9e3a-47e6-9543-effa89bbb6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:53.854681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:53.889574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 680/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Jul 21 12:07:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:53.897629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:53.899280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:54.040414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8c2d0-43bd-42f7-95bd-4d3500b2ccf0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:54.090239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:54.128590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:54.129354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:54.129614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:54.129822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:54.132141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:54.277558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:54.279046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a141e-41f4-47b0-be19-bcd4f45d37ff service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:54.301152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 693/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:55.044985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:55.045573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:55.045573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:55.046892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 681/1374] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:55.058380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af978-5a57-49e6-aa09-46b33987e3f5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:55.058796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af978-5a57-49e6-aa09-46b33987e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:55.059118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af978-5a57-49e6-aa09-46b33987e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:55.131381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ed1af978-5a57-49e6-aa09-46b33987e3f5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:55.132665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ed1af978-5a57-49e6-aa09-46b33987e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:55.154111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:55.234036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 694/1375] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:55.279381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:55.281032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:55.429465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48674b3b-c125-4c9d-b41c-02b58298ec2a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:55.490432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:55.560639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:55.561296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:55.561578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:55.561882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:55.563662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:55.661483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:55.663110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c686ab1-b749-4099-bdad-0fb1c7e8b2c8 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:55.683241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 682/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:07:57.816155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:57.816854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:57.816854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:57.818167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 695/1377] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:07:57.824011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209010-f743-46d3-ae06-1c275b16d2a4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:57.824475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209010-f743-46d3-ae06-1c275b16d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:57.824583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79209010-f743-46d3-ae06-1c275b16d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:57.913796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-79209010-f743-46d3-ae06-1c275b16d2a4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:07:57.915020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-79209010-f743-46d3-ae06-1c275b16d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:57.922914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:58.044825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 683/1378] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:07:58.091724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:07:58.093707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:58.265281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d668a-8267-460e-87a2-a07ddb678728 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:07:58.269733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:07:58.431358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:07:58.432853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:07:58.433179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:07:58.433649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:07:58.436043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:07:58.580944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:07:58.584678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1323b4a-d9b8-4c68-b89c-5398eb99420d service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:07:58.621427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 696/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:01.037328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47038e58-859b-4ea4-977d-4611371a451e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:01.038741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47038e58-859b-4ea4-977d-4611371a451e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:01.039230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47038e58-859b-4ea4-977d-4611371a451e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:01.042043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 684/1380] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:01.051116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c956e-23e3-47ac-98a3-5b85bc9b8239 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:01.051663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c956e-23e3-47ac-98a3-5b85bc9b8239 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:01.052125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c956e-23e3-47ac-98a3-5b85bc9b8239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:01.071339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:01.187702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-4b3c956e-23e3-47ac-98a3-5b85bc9b8239 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:01.190172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4b3c956e-23e3-47ac-98a3-5b85bc9b8239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:01.389525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 697/1381] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:01.434963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47038e58-859b-4ea4-977d-4611371a451e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:01.436710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-47038e58-859b-4ea4-977d-4611371a451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:01.593085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:01.606536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47038e58-859b-4ea4-977d-4611371a451e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:01.747212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:01.748047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:01.748455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:01.748801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:01.750566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:01.859669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:01.861315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fbb94-79e1-42fe-ad70-5f9afb35da56 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:01.873090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:01.895192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 685/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:04.137303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:04.137983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:04.137983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:04.139404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 698/1383] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:04.145450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0ef38-591b-480f-9448-b35460082bd4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:04.146017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0ef38-591b-480f-9448-b35460082bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:04.146017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0ef38-591b-480f-9448-b35460082bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:04.223444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-15a0ef38-591b-480f-9448-b35460082bd4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:04.224804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-15a0ef38-591b-480f-9448-b35460082bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:04.347138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 686/1384] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:04.394543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:04.396071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:04.496857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:04.572512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30212-7a0d-4915-9dab-92807e8ce830 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:04.701905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:04.702613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:04.703203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:04.703498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:04.705271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:04.761964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:04.816935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:04.818771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7f051-72a7-4d4e-888a-b2aaf2bdbd35 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:04.842018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 699/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:07.032890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:07.033718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:07.033718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:07.035450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 687/1386] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:08:07.043007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3982-a4e1-4037-8d50-eac77e61eaba None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:07.043983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3982-a4e1-4037-8d50-eac77e61eaba None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:07.043983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3982-a4e1-4037-8d50-eac77e61eaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:07.120891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-781f3982-a4e1-4037-8d50-eac77e61eaba None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:07.121995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-781f3982-a4e1-4037-8d50-eac77e61eaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:07.236134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 700/1387] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:07.295163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:07.297160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:07.392874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:07.477411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d781382-eb72-4b4b-b560-f5e98570bed5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:07.607640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:07.608405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:07.608715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:07.609012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:07.610871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:07.657786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:07.725620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:07.727561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57837f-8fbf-49e0-b953-cc0480793451 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:07.749361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 688/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:10.080074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:10.081175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:10.081175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:10.083150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 701/1389] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:08:10.090814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54659-b63d-4d9b-b827-3ea70a2a1fe3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:10.091428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54659-b63d-4d9b-b827-3ea70a2a1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:10.091568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e54659-b63d-4d9b-b827-3ea70a2a1fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:10.185412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-10e54659-b63d-4d9b-b827-3ea70a2a1fe3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:10.187125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-10e54659-b63d-4d9b-b827-3ea70a2a1fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:10.328551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 689/1390] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:10.377749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:10.380199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:10.422348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:10.555590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f7854-9e6f-4f1d-b43e-b27fc933b663 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:10.684847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:10.702356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:10.703651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:10.704031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:10.704405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:10.706873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:10.827361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:10.829963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda734-443e-4c6f-bfdc-2ef24142a16f service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:10.855781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 702/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:08:13.026366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.026979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.026979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.028399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 690/1392] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:13.034360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc9ad-a5ff-48d7-97d1-67a6d9bade82 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.034990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc9ad-a5ff-48d7-97d1-67a6d9bade82 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.035086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fc9ad-a5ff-48d7-97d1-67a6d9bade82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.113852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-cc0fc9ad-a5ff-48d7-97d1-67a6d9bade82 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:13.115413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cc0fc9ad-a5ff-48d7-97d1-67a6d9bade82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:13.201629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:13.246115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 703/1393] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:13.293043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:13.294824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:13.310525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:13.497616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4599895-321b-43fa-8763-e8bdbd6514a0 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:13.594162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:13.624388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:13.625140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.625522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.625802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.627285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e10cd-6826-42ae-a49e-95df4c6ad9d0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.627592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:13.627711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e10cd-6826-42ae-a49e-95df4c6ad9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.627869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e10cd-6826-42ae-a49e-95df4c6ad9d0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.631525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 704/1394] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:13.635554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c3d0e-c385-4b3f-af4a-e9ed64c3e109 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.635837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c3d0e-c385-4b3f-af4a-e9ed64c3e109 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.636119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c3d0e-c385-4b3f-af4a-e9ed64c3e109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.709281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d08c3d0e-c385-4b3f-af4a-e9ed64c3e109 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:13.710612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d08c3d0e-c385-4b3f-af4a-e9ed64c3e109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:13.731717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:13.733539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443262f-c2b1-46be-81c5-131644dcf6fb service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:13.754578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 691/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:13.828014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 705/1396] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:13.890264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.890580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.890832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.892445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 692/1397] 10.222.0.18 () {54 vars in 966 bytes} [Thu Jul 21 12:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:13.898678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e23823-b72b-47cd-84be-c25329eb000a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:13.898950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e23823-b72b-47cd-84be-c25329eb000a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:13.899249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e23823-b72b-47cd-84be-c25329eb000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:13.976198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:14.020399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-10e23823-b72b-47cd-84be-c25329eb000a None None] MFA Rules not processed for user `f54544af7d84443e9b8fdbcb718bf824`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:14.022479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-10e23823-b72b-47cd-84be-c25329eb000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:14.128274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 706/1398] 10.222.0.18 () {58 vars in 1063 bytes} [Thu Jul 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:14.136089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:14.137639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:14.262824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:14.264600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ef6c1-7be5-49b0-8e33-9bc172785124 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:14.364242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:14.364810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:14.365092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:14.365338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:14.367013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:14.513511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:14.514992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb4b87-d123-4193-b529-08c7e91a7097 service glance] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:14.536766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 693/1399] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:15.305323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adade-c09e-4251-9be2-178f3582318c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:15.305885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adade-c09e-4251-9be2-178f3582318c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:15.305885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adade-c09e-4251-9be2-178f3582318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:15.386352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-735adade-c09e-4251-9be2-178f3582318c None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:15.387615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-735adade-c09e-4251-9be2-178f3582318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:15.478304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 707/1400] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:15.523801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:15.524376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:15.524474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:15.526017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 694/1401] 10.222.0.18 () {54 vars in 943 bytes} [Thu Jul 21 12:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:15.532816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50a5bd-34b1-4d56-a2f1-32317934eff6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:15.533207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50a5bd-34b1-4d56-a2f1-32317934eff6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:15.533487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50a5bd-34b1-4d56-a2f1-32317934eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:15.537107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:15.615001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ef50a5bd-34b1-4d56-a2f1-32317934eff6 None None] MFA Rules not processed for user `d14dd264bd4343b49ef968789de08f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:15.616642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ef50a5bd-34b1-4d56-a2f1-32317934eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:15.736811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 708/1402] 10.222.0.18 () {58 vars in 1040 bytes} [Thu Jul 21 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:15.747334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:15.748923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:15.790094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:15.910465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4847ca-5c2c-4419-8ff7-4d07a090c90a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:15.910849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4847ca-5c2c-4419-8ff7-4d07a090c90a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:15.911063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4847ca-5c2c-4419-8ff7-4d07a090c90a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:15.911446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b75d28-4cae-4c35-a375-8fc0d77666ef None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:15.912534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 709/1403] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:15.918175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:15.918565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:15.918873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:15.991565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:15.992826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:16.007290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:16.024420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:16.025101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:16.025399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:16.025653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:16.027571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:16.043420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:16.102944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 710/1404] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:16.125620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:16.127150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478167f2-338b-4890-acc5-41d8ebd8ad76 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:16.136638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 695/1405] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:08:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:08:16.150476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:16.151718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:16.314868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1e0a-5d4c-45b8-a3ec-cf9a6ccb8e87 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:16.351052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:16.484211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:16.485669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:16.486295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:16.486888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:16.489874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:16.610452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:16.612106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df0df5-bf16-48ea-9dc7-3cc7f78555eb service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:16.636775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 711/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:19.071581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:19.072551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:19.072551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:19.073599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 696/1407] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:19.080888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9210-c630-45a9-843e-bc0ef73fc8d6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:19.081477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9210-c630-45a9-843e-bc0ef73fc8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:19.081477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9210-c630-45a9-843e-bc0ef73fc8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:19.145478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:19.200360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8bfc9210-c630-45a9-843e-bc0ef73fc8d6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:19.202002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8bfc9210-c630-45a9-843e-bc0ef73fc8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:19.395505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 712/1408] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:19.461165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:19.463930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:19.741169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc24b35-2bfc-4ca3-95ee-1dcb6f5f1cbf None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:19.764803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:19.930164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:19.931452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:19.931962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:19.932428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:19.935429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:20.098564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:20.100342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd30fc5-bfe9-4ab1-ba2c-09df3f1accd3 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:20.125677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 697/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:29.379677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=50 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:30.123097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:38.676218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64dec-8234-4ec8-8793-0e6f45bcbe0b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:38.676836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64dec-8234-4ec8-8793-0e6f45bcbe0b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:38.676836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c64dec-8234-4ec8-8793-0e6f45bcbe0b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:38.678140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 713/1410] 10.222.0.18 () {54 vars in 918 bytes} [Thu Jul 21 12:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:38.683672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b8d62-9982-497e-b7f9-6c85f9b09a09 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:38.684171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b8d62-9982-497e-b7f9-6c85f9b09a09 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:38.684171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b8d62-9982-497e-b7f9-6c85f9b09a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:38.761672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-571b8d62-9982-497e-b7f9-6c85f9b09a09 None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:38.762905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-571b8d62-9982-497e-b7f9-6c85f9b09a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:38.873645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 698/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jul 21 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:38.888886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722542-b274-4a91-9f34-2ea70f9f6ad4 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:38.889171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722542-b274-4a91-9f34-2ea70f9f6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:38.889444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722542-b274-4a91-9f34-2ea70f9f6ad4 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:38.890840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 714/1412] 10.222.0.18 () {54 vars in 918 bytes} [Thu Jul 21 12:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:38.895897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843ec-821b-476c-ae94-937996986c23 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:38.896162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843ec-821b-476c-ae94-937996986c23 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:38.896651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843ec-821b-476c-ae94-937996986c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:38.970246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:38.973216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-948843ec-821b-476c-ae94-937996986c23 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:38.974912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-948843ec-821b-476c-ae94-937996986c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:39.098077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 699/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jul 21 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:41.622725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:41.623267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:41.623267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:41.624650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:41.630459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ee69-f5fd-49d2-9722-c83c65ab6079 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:41.630938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ee69-f5fd-49d2-9722-c83c65ab6079 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:41.631044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ee69-f5fd-49d2-9722-c83c65ab6079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:41.706763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1f29ee69-f5fd-49d2-9722-c83c65ab6079 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:41.708047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1f29ee69-f5fd-49d2-9722-c83c65ab6079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:41.772851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:41.822749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:41.868346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:41.870056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:42.041686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882bef5-a761-4d5f-a650-9406080fa136 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:42.124955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:42.170324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:42.170983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:42.171261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:42.171524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:42.173227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:42.267956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:42.269576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659dc1-96f8-4945-b1ef-0c7c8be312cd service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:42.288859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 716/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:43.203814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:43.204376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:43.204376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:43.205794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:43.211038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd2e3-3eda-4117-9cf5-12332315bbc8 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:43.211454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd2e3-3eda-4117-9cf5-12332315bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:43.211583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd2e3-3eda-4117-9cf5-12332315bbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:43.289869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b80bd2e3-3eda-4117-9cf5-12332315bbc8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:43.291147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b80bd2e3-3eda-4117-9cf5-12332315bbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:43.319999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:43.392981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:43.541406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:43.542835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:43.694896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaecbea-059d-4928-9e0d-6f223985cf28 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:43.736250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:43.809064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:43.809614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:43.809888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:43.810157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:43.811863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:43.909060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:43.910749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a09ff6-4894-4a0b-8e46-84a0ca68a61c service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:43.931035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 702/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:44.839589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:44.840159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:44.840159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:44.841531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:44.846921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056004f-d5af-472b-b393-236baaeec961 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:44.847401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056004f-d5af-472b-b393-236baaeec961 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:44.847526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056004f-d5af-472b-b393-236baaeec961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:44.903399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:44.929553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-6056004f-d5af-472b-b393-236baaeec961 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:44.931160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6056004f-d5af-472b-b393-236baaeec961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:45.035125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:45.186443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:45.188096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:45.346870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a22c6b-0694-452c-9a16-095452f10b46 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:45.361314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:45.469479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:45.470057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:45.470333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:45.470623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:45.472471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:45.566623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:45.568325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fae846-37ad-4a93-a15e-cd8d485b9497 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:45.587536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 719/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:46.471734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:46.472290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:46.472290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:46.473649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:46.479634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596b91-d936-4fd4-8e8b-104f164c1fb2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:46.480191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596b91-d936-4fd4-8e8b-104f164c1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:46.480191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7596b91-d936-4fd4-8e8b-104f164c1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:46.499959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:46.560258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e7596b91-d936-4fd4-8e8b-104f164c1fb2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:46.561473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e7596b91-d936-4fd4-8e8b-104f164c1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:46.659990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:46.810928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:46.812858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:46.933024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:46.972969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df64dff-3a71-4662-9df4-c4fc9a4b3434 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:47.087021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:47.087021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:47.087636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:47.087636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:47.089157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:47.171183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:47.185093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:47.186815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca118a0-1364-490d-91dc-1c4875625373 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:47.214668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 705/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:48.095011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:48.095759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:48.095759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:48.096998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:08:48.103502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77727665-c0d4-4d9e-9652-fb91992710f6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:48.104077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77727665-c0d4-4d9e-9652-fb91992710f6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:48.104077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77727665-c0d4-4d9e-9652-fb91992710f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:48.187021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-77727665-c0d4-4d9e-9652-fb91992710f6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:48.188308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-77727665-c0d4-4d9e-9652-fb91992710f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:48.290969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:48.440048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:48.441605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:48.534253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:48.601441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c41877-4135-453f-8eb4-51659b41af78 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:48.728671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:48.729231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:48.729513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:48.729772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:48.731465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:48.782020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:48.825389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:48.827396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60f1a1-e08f-45b5-b2c0-15cf9f2a831c service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:48.846205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 722/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:49.711019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:49.711644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:49.711644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:49.713029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:49.718761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba1db-a36a-4872-a38a-6d518ee5ebaf None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:49.719253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba1db-a36a-4872-a38a-6d518ee5ebaf None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:49.719375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba1db-a36a-4872-a38a-6d518ee5ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:49.814675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-efbba1db-a36a-4872-a38a-6d518ee5ebaf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:49.815959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-efbba1db-a36a-4872-a38a-6d518ee5ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:49.918020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:50.073766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:50.075353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:50.130339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:50.237058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e310e17-875e-4e88-916d-576b92337d22 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:50.357388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:50.357959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:50.358234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:50.358500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:50.360205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:50.368291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:50.458692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:50.460394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6988e2b-4bde-4c93-9112-79bc255bd690 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:50.480195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 708/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:51.287092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:51.287663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:51.287663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:51.288902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:51.294386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6803e-94ae-4dbf-a6eb-632ced429b09 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:51.294847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6803e-94ae-4dbf-a6eb-632ced429b09 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:51.294951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6803e-94ae-4dbf-a6eb-632ced429b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:51.378522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bda6803e-94ae-4dbf-a6eb-632ced429b09 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:51.379854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bda6803e-94ae-4dbf-a6eb-632ced429b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:51.452895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:51.486025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:08:51.632805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:51.634274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:51.652787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:51.790670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ef47-d372-4a06-93d9-306fb182c843 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:51.880789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:51.907271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:51.907959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:51.908234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:51.908507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:51.910162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:52.002432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:52.004093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6111e7-ae5f-4656-8cec-41fb2ab3457d service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:52.022244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 725/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:52.832435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:52.833049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:52.833049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:52.834387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:52.840971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005caa7-b9c1-4212-87a4-1c44550656f3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:52.841505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005caa7-b9c1-4212-87a4-1c44550656f3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:52.841604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005caa7-b9c1-4212-87a4-1c44550656f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:52.926316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1005caa7-b9c1-4212-87a4-1c44550656f3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:52.927555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1005caa7-b9c1-4212-87a4-1c44550656f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:52.972626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:53.030071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:53.180813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:53.182362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:53.334217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c345f9-17b9-461a-aee7-a0a94aa5a849 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:53.391001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:53.463816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:53.464359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:53.464642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:53.464917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:53.466606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:53.557168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:53.558787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d68516-b951-4271-8af9-f52bdfdf6201 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:53.577994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 711/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:54.389857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:54.390449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:54.390449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:54.391807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:54.398195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d21c-074b-4ee0-b913-fb9f3c5ebabd None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:54.398783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d21c-074b-4ee0-b913-fb9f3c5ebabd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:54.398886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d21c-074b-4ee0-b913-fb9f3c5ebabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:54.495891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e3b7d21c-074b-4ee0-b913-fb9f3c5ebabd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:54.497044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e3b7d21c-074b-4ee0-b913-fb9f3c5ebabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:54.499143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:54.602907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:54.755819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:54.757264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:54.912039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b969f-fd08-412e-a653-197dd8967230 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:54.940817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:55.032897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:55.033460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:55.033739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:55.034012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:55.035859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:55.130756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:55.132412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512d03-c84b-4925-8ee1-17096d1a0c31 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:55.152966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 728/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:57.071743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:57.072277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:57.072277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:57.073675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:57.080625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291849d-265a-4638-afb3-42aec56cd0b1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:57.081139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291849d-265a-4638-afb3-42aec56cd0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:57.081240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291849d-265a-4638-afb3-42aec56cd0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:57.135670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:57.166168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3291849d-265a-4638-afb3-42aec56cd0b1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:57.167401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3291849d-265a-4638-afb3-42aec56cd0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:57.269008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:08:57.314773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:08:57.316447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:57.426779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c07bbb-0897-46fd-8aba-f931eaae480a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:08:57.464288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:08:57.517302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:08:57.518004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:57.518283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:57.518550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:57.520376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:57.645637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:08:57.647091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2e18-19ea-4338-b567-1245a1e08e24 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:08:57.666697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 714/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:08:59.874574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:59.875186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:59.875186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:59.876549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:08:59.882244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b01b-a41c-49f7-bf3d-b4ea87b24f22 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:08:59.882747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b01b-a41c-49f7-bf3d-b4ea87b24f22 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:08:59.882854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b01b-a41c-49f7-bf3d-b4ea87b24f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:08:59.958536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4796b01b-a41c-49f7-bf3d-b4ea87b24f22 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:08:59.959816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4796b01b-a41c-49f7-bf3d-b4ea87b24f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:08:59.975637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:00.061518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:00.103577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:00.105213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:00.220335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d465f-b43e-4255-a450-fa09524f9297 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:00.235276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:00.305925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:00.306580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:00.306854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:00.307126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:00.308858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:00.439011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:00.440420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973c59a-599a-4eb8-9f3a-3a914d0d7cce service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:00.461460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 731/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:02.808590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:02.809428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:02.809428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:02.811442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:02.818634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651d05-1d0e-4c1e-8adc-b194cac19e00 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:02.819131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651d05-1d0e-4c1e-8adc-b194cac19e00 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:02.819249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651d05-1d0e-4c1e-8adc-b194cac19e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:02.877340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:02.901844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7e651d05-1d0e-4c1e-8adc-b194cac19e00 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:02.903145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7e651d05-1d0e-4c1e-8adc-b194cac19e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:03.012256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:03.052531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:03.054086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:03.174774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5be65-ce45-4219-9abc-1d1cd128f257 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:03.216901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:03.263663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:03.264395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:03.264725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:03.265073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:03.268060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:03.391439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:03.392934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6d9ef-65d2-462f-a228-82038585f288 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:03.414655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 717/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:05.604530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:05.605182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:05.605182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:05.606543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:05.612415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a64a32-2415-4a8d-b4e0-4e54819ad021 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:05.612883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a64a32-2415-4a8d-b4e0-4e54819ad021 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:05.612997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a64a32-2415-4a8d-b4e0-4e54819ad021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:05.684654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a2a64a32-2415-4a8d-b4e0-4e54819ad021 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:05.685874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a2a64a32-2415-4a8d-b4e0-4e54819ad021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:05.715861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:05.797424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:05.837653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:05.839256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:05.968756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc6cf-741a-47c8-a325-6088fceb6093 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:05.996114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:06.053879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:06.054539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:06.054847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:06.055111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:06.056802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:06.170838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:06.172348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2449e2e-2129-4031-8b90-319a28ee49f3 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:06.191764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 734/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:08.394546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:08.395209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:08.395332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:08.396720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:08.408472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c642-2bda-42b4-ad94-9ce46ed300ff None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:08.409096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c642-2bda-42b4-ad94-9ce46ed300ff None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:08.409205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0c642-2bda-42b4-ad94-9ce46ed300ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:08.477279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:08.480943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-add0c642-2bda-42b4-ad94-9ce46ed300ff None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:08.482784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-add0c642-2bda-42b4-ad94-9ce46ed300ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:08.583995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:08.626605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:08.628403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:08.748500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df538b-3722-477a-a071-d46ed3ed02b4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:08.801947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:08.828513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:08.829178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:08.829465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:08.829735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:08.831420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:08.968147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:08.969654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dd2de-1fb4-4e69-a308-e0d1de076144 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:08.991409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 720/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:11.111950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:11.112546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:11.112546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:11.114127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:11.124770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b96d0-3f9f-4c56-b568-7133456f0e1c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:11.125332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b96d0-3f9f-4c56-b568-7133456f0e1c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:11.125332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b96d0-3f9f-4c56-b568-7133456f0e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:11.203021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ea4b96d0-3f9f-4c56-b568-7133456f0e1c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:11.204327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ea4b96d0-3f9f-4c56-b568-7133456f0e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:11.239772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:11.313367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:11.355140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:11.356584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:11.471980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe8af7-7250-4914-bdab-3d262c2470c3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:11.502678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:11.550892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:11.551560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:11.551834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:11.552104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:11.553749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:11.672792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:11.674272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cee3f-7cd8-4a9f-a62c-21831059ff57 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:11.695829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 737/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:13.790513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:13.791134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:13.791134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:13.792508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:13.798228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f8b4-b2fb-48c7-81fb-b0b9c6d93053 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:13.798761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f8b4-b2fb-48c7-81fb-b0b9c6d93053 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:13.798761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450f8b4-b2fb-48c7-81fb-b0b9c6d93053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:13.871172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e450f8b4-b2fb-48c7-81fb-b0b9c6d93053 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:13.872453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e450f8b4-b2fb-48c7-81fb-b0b9c6d93053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:13.881196 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:13.980455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:14.020583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:14.022215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:14.139456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca781-8bb5-4bd2-825b-eaafe76236c4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:14.213142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:14.234516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:14.235178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:14.235454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:14.235754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:14.237426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:14.364106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:14.365626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d03363-5ce6-4ae4-80b7-3140c12a04b4 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:14.386674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 723/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:16.485842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:16.486463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:16.486463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:16.487785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:16.493975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757dda-df80-4249-833f-f99e29d0007f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:16.494450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757dda-df80-4249-833f-f99e29d0007f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:16.494544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757dda-df80-4249-833f-f99e29d0007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:16.575131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-90757dda-df80-4249-833f-f99e29d0007f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:16.577469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-90757dda-df80-4249-833f-f99e29d0007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:16.620796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:16.687023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:16.726032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:16.727626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:16.853031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ca001-b9c0-4924-88d1-8cfec9a159ed None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:16.895272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:16.943227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:16.944412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:16.944412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:16.944412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:16.946108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:17.069417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:17.070951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cbdad-8548-41db-bbb7-5cdbf5d4d8fe service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:17.093661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 740/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:19.167696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:19.168528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:19.168741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:19.170358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:19.179904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd405-8493-4283-ab3c-5ecb98a4e216 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:19.180403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd405-8493-4283-ab3c-5ecb98a4e216 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:19.180511 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd405-8493-4283-ab3c-5ecb98a4e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:19.250773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-107cd405-8493-4283-ab3c-5ecb98a4e216 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:19.252004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-107cd405-8493-4283-ab3c-5ecb98a4e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:19.270633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:19.355240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:09:19.393159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:19.394802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:19.511374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d285c1-c9e0-47e1-a00b-f27603398ab7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:19.586032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:19.593609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:19.594268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:19.594549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:19.594820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:19.596525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:19.723462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:19.724989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52dddb-3938-402e-910f-0c65f187117b service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:19.747877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 726/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:21.837623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:21.838284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:21.838284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:21.839549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:21.845531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579a2d-06f9-45e1-845a-2004f117e2ab None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:21.846010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579a2d-06f9-45e1-845a-2004f117e2ab None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:21.846128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a579a2d-06f9-45e1-845a-2004f117e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:21.918812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9a579a2d-06f9-45e1-845a-2004f117e2ab None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:21.920099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9a579a2d-06f9-45e1-845a-2004f117e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:21.989992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:22.042665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:22.082650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:22.084295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:22.199874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01d5a3-80bf-4101-a3ff-942900e95aac None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:22.250536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:22.278588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:22.279242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:22.279506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:22.279924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:22.281775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:22.421612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:22.422990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e35f15-4a6c-4560-b8f2-79073198e8e5 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:22.445584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 743/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:32.040737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=22 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:32.442716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=66 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:42.056427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac4e3-edd9-481c-af6f-f740f0e31fdc None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:42.057053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac4e3-edd9-481c-af6f-f740f0e31fdc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:42.057053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bac4e3-edd9-481c-af6f-f740f0e31fdc None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:42.058366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 728/1471] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:42.065384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b767-96cb-441c-9baa-baa2b729cdb7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:42.065969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b767-96cb-441c-9baa-baa2b729cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:42.065969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b767-96cb-441c-9baa-baa2b729cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:42.138359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a8f8b767-96cb-441c-9baa-baa2b729cdb7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:42.139598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a8f8b767-96cb-441c-9baa-baa2b729cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:42.246348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 744/1472] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:42.350938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:42.351226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:42.351554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:42.353324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 729/1473] 10.222.0.18 () {54 vars in 966 bytes} [Thu Jul 21 12:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:42.360716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273166-fc8b-4f2a-ab63-b9529ba7f9e2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:42.361050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273166-fc8b-4f2a-ab63-b9529ba7f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:42.361489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273166-fc8b-4f2a-ab63-b9529ba7f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:42.452097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e3273166-fc8b-4f2a-ab63-b9529ba7f9e2 None None] MFA Rules not processed for user `f54544af7d84443e9b8fdbcb718bf824`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:42.453347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e3273166-fc8b-4f2a-ab63-b9529ba7f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:42.475967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:42.535216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 745/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Thu Jul 21 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:42.542873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:42.544440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:42.652853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf94c-fcbd-480f-b012-70ef239f2036 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:42.729259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:42.729921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:42.730203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:42.730483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:42.732184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:42.787932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:42.847112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:42.848606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b678e-feb3-40b1-811f-1e33e934c2e0 service glance] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:42.868005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 730/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:09:44.991354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:44.991791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:44.991928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:44.993988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 746/1476] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:45.001675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239de69-4dc7-43b3-ae55-522dc45232ae None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:45.002230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239de69-4dc7-43b3-ae55-522dc45232ae None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:45.002230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239de69-4dc7-43b3-ae55-522dc45232ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:45.073616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b239de69-4dc7-43b3-ae55-522dc45232ae None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:45.074858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b239de69-4dc7-43b3-ae55-522dc45232ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:45.185039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 731/1477] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:45.210519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:45.212129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:45.357180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f0da5-e28c-4e1f-a42d-6b4c72856720 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:45.411737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:45.462420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:45.463107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:45.463385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:45.463680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:45.464419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:45.466418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:09:45.466418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:09:45.466418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:09:45.467817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d810acb-9e20-4c18-8394-b8cc5433da09 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:45.491190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 747/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Thu Jul 21 12:09:45 2022] GET /identity/v3/projects => generated 2553 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:09:47.538440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:47.539052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:47.539052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:47.540394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 732/1479] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:47.546679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7d34-c22e-4b80-9c13-938baeb904cc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:47.547171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7d34-c22e-4b80-9c13-938baeb904cc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:47.547266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c7d34-c22e-4b80-9c13-938baeb904cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:47.619939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-331c7d34-c22e-4b80-9c13-938baeb904cc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:47.621191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-331c7d34-c22e-4b80-9c13-938baeb904cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:47.728426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 748/1480] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:47.777063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:47.777063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:47.787060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:47.949406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8863df-0c6d-4885-8d0a-319d1c97f981 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:48.048166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:48.083378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:48.084068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:48.084332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:48.084627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:48.086296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:48.179156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:48.180807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e6f40-9f3c-4009-a81c-a4647068fc33 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:48.199587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 733/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:50.435925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:50.436469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:50.436469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:50.437767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 749/1482] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:50.443622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248ef3b-1ca6-48aa-9802-a90ececa2617 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:50.444113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248ef3b-1ca6-48aa-9802-a90ececa2617 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:50.444230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248ef3b-1ca6-48aa-9802-a90ececa2617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:50.517377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2248ef3b-1ca6-48aa-9802-a90ececa2617 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:50.518612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2248ef3b-1ca6-48aa-9802-a90ececa2617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:50.567643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:50.624581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 734/1483] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:50.670607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:50.672203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:50.697181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:50.786822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb26f8-153d-4310-a3eb-4e89e25b350d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:50.869021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:50.869683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:50.869957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:50.870225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:50.871974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:50.956816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:50.994371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:50.995886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fde5ff-4377-4e5c-8b18-3698614c333b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:51.014653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 750/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:53.082138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:53.083190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:53.083190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:53.084919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 735/1485] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:53.090953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce97799-dce5-4f91-acf7-1e66b3bdabc3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:53.091444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce97799-dce5-4f91-acf7-1e66b3bdabc3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:53.091574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce97799-dce5-4f91-acf7-1e66b3bdabc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:53.163363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8ce97799-dce5-4f91-acf7-1e66b3bdabc3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:53.164707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8ce97799-dce5-4f91-acf7-1e66b3bdabc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:53.274572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 751/1486] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:53.321920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:53.323533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:53.442903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aebf2a-fffa-4d49-95e4-c233dda3dfd2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:53.528127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:53.530350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:53.531078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:53.531379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:53.531707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:53.533988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:53.659605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:53.661078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f0767-8ff5-4ee8-a2ba-60d7b68096b3 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:53.679377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 736/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:55.863206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:55.864177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:55.864177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:55.865912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 752/1488] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:55.874118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4056-6b1c-4cd5-b441-08969d83666b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:55.874619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4056-6b1c-4cd5-b441-08969d83666b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:55.874734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4056-6b1c-4cd5-b441-08969d83666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:55.948996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-342d4056-6b1c-4cd5-b441-08969d83666b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:55.950237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-342d4056-6b1c-4cd5-b441-08969d83666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:56.010001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:56.056690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 737/1489] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:56.098647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:56.100266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:56.143011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:56.224021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d4b3d-37c7-4446-8514-e74a12f001eb None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:56.302166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:56.302841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:56.303125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:56.303404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:56.305148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:56.402469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:56.437010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:56.438637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ebeaa-987d-4e31-ba7f-41d719b5bf71 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:56.461359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 753/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:09:58.772754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:58.773384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:58.773384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:58.774712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 738/1491] 10.222.0.18 () {54 vars in 926 bytes} [Thu Jul 21 12:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:09:58.780777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc4e2-1beb-49e8-9a1f-594d53944912 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:58.781627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc4e2-1beb-49e8-9a1f-594d53944912 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:58.781848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc4e2-1beb-49e8-9a1f-594d53944912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:58.856742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8e2fc4e2-1beb-49e8-9a1f-594d53944912 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:09:58.857996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8e2fc4e2-1beb-49e8-9a1f-594d53944912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:58.969191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 754/1492] 10.222.0.18 () {58 vars in 1023 bytes} [Thu Jul 21 12:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:09:59.010487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:09:59.012188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:59.128109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941ed8f-46ff-4c20-8d73-25a679d12db1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:09:59.224411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:09:59.225106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:09:59.225390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:09:59.225668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:09:59.227516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:09:59.239308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:09:59.350458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:09:59.351971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe502883-27e0-4ee1-858e-1b34f784b418 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:09:59.371635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 739/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:10:08.967148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=82 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:10:09.369038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=50 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:41.634817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:41.635442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:41.635442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:41.721093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:11:41.722340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:41.831984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 755/1494] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:11:41.841285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:41.841831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:41.841831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:41.925607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:11:41.927375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:42.031500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 740/1495] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:11:42.036464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:42.037697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:42.049204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934f1ca-5171-48a8-b499-7b5f49bb471f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:42.116082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:42.154386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:42.155073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:42.155344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:42.155649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:42.156437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:11:42.156740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:42.157776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:42.157776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:11:42.157776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:11:42.158683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:42.171143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 756/1496] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:11:42.181652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:42.191710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dca90-0cfe-4be0-9c79-4a72dea177f2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:42.258320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:42.296399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:42.297079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:42.297361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:42.297633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:42.298369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:42.300201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:42.300201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:11:42.300201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:11:42.301570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:42.497536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 741/1497] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:11:42 2022] POST /identity/v3/projects => generated 377 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:11:42.501954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:42.503570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:42.538562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fca6b-cd03-42c7-a6d1-574413c9f1b3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:42.639761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:42.640478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:42.640709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:42.640983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:42.641679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:42.642710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:42.642710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:11:42.642710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:11:42.643954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:42.689383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 757/1498] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:11:42 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:11:42.693853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:42.695442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:42.770030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f43a27-98be-4470-ae5d-9959fec52c61 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:42.886433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:42.899859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:42.900559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:42.900840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:42.901122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:42.902044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:42.903103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:42.903103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:11:42.903103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:11:42.904059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:42.918486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 742/1499] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:11:42.923017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:42.924705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:42.937182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54729-deba-4376-87cf-3149f4b85188 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:42.967425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:43.043684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:43.044226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.044506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.044776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.045509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:43.046532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:43.046532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:11:43.046532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:11:43.047426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bf801-7889-4e3d-b020-3ee029d21720 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:43.060848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 758/1500] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:11:43.065398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:43.066961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:43.115000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be34c59-8b1f-4715-8323-8148f97cbac4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:43.226429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:43.227344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.227660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.227943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] PATH_INFO: `/v3/projects/5f88f92a6f9f421d949e8c769b637b16/users/fc6d7f6ccac74896a392747b73b563b1/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.251868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f88f92a6f9f421d949e8c769b637b16, user_id=fc6d7f6ccac74896a392747b73b563b1, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:43.252922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:43.252922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:11:43.252922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:11:43.254902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc39699-dda3-44ab-a2fa-f62ab3817210 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:43.281943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 743/1501] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:11:43 2022] PUT /identity/v3/projects/5f88f92a6f9f421d949e8c769b637b16/users/fc6d7f6ccac74896a392747b73b563b1/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:11:43.294763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a6d5-2e11-4407-981c-b499d2c83796 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.294977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a6d5-2e11-4407-981c-b499d2c83796 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.295221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a6d5-2e11-4407-981c-b499d2c83796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.362465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2ae2a6d5-2e11-4407-981c-b499d2c83796 None None] MFA Rules not processed for user `fc6d7f6ccac74896a392747b73b563b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:11:43.364278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2ae2a6d5-2e11-4407-981c-b499d2c83796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:43.454423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 759/1502] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:11:43.472973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.473237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.473518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.474897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 744/1503] 10.222.0.18 () {54 vars in 964 bytes} [Thu Jul 21 12:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:11:43.480176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff2a55-4f88-47cb-b593-6393ee5b6501 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.480463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff2a55-4f88-47cb-b593-6393ee5b6501 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.480741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff2a55-4f88-47cb-b593-6393ee5b6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.512424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:43.559553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f7ff2a55-4f88-47cb-b593-6393ee5b6501 None None] MFA Rules not processed for user `5493d9cde9404225bedc423f0ecf213c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:11:43.560680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f7ff2a55-4f88-47cb-b593-6393ee5b6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:43.638744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 760/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Thu Jul 21 12:11:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:11:43.647416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:43.649238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:43.707424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:43.771356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbae5dc-f74c-474a-a54a-50545e10ace7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:43.861182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:43.861938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:43.862220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:43.862493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:43.864352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:43.980852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:43.982327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:43.986385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:44.005532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 745/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:11:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:11:44.053603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40260b-9976-4511-83bc-0a05ca10f5f3 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:44.054187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40260b-9976-4511-83bc-0a05ca10f5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:44.054187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40260b-9976-4511-83bc-0a05ca10f5f3 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:44.055535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 761/1506] 10.222.0.18 () {52 vars in 829 bytes} [Thu Jul 21 12:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:11:44.060236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:44.061621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:44.181526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ad30-4824-4093-98ac-e9175c340582 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:44.293432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:44.294388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:44.294671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:44.294945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] PATH_INFO: `/v3/users/fc6d7f6ccac74896a392747b73b563b1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:44.310638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6d7f6ccac74896a392747b73b563b1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:44.311734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:44.311734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:11:44.311734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:11:44.312976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:44.324535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:44.384531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6d7f6ccac74896a392747b73b563b1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:44.394463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6d7f6ccac74896a392747b73b563b1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:44.422856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6d7f6ccac74896a392747b73b563b1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:44.423256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6d7f6ccac74896a392747b73b563b1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:44.437170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6d7f6ccac74896a392747b73b563b1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:44.443603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 746/1507] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:11:44 2022] DELETE /identity/v3/users/fc6d7f6ccac74896a392747b73b563b1 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:11:44.449023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:44.449309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:44.449587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:44.516078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:11:44.517319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:44.607534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:44.621831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 762/1508] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:11:44.632148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:44.633472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:44.745447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5f8e-864e-44ab-a49e-3ca8544f2eb8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:44.804761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:44.834752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:44.835432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:44.835746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:44.836020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:44.837755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:44.958003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:44.959772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeec459-e310-4084-9cb9-4d341fc481a7 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:44.980013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 747/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:11:45.396314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:11:45.397618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:11:45.411168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da8785-b0ef-4dd5-bce7-cb07c3efdbce None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:11:45.529472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:11:45.530393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:11:45.530667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:11:45.530936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] PATH_INFO: `/v3/projects/5f88f92a6f9f421d949e8c769b637b16` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:11:45.542990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f88f92a6f9f421d949e8c769b637b16)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:11:45.544151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:11:45.544151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:11:45.544151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:11:45.545364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:11:45.612425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f88f92a6f9f421d949e8c769b637b16', 'request_id': 'req-1009453c-1f80-473b-bd2b-8862ddadad8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:45.627036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f88f92a6f9f421d949e8c769b637b16', 'request_id': 'req-1009453c-1f80-473b-bd2b-8862ddadad8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:45.634414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f88f92a6f9f421d949e8c769b637b16', 'request_id': 'req-1009453c-1f80-473b-bd2b-8862ddadad8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:45.666461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] The token cache is being invalidate because project 5f88f92a6f9f421d949e8c769b637b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:11:45.666746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:45.667183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1009453c-1f80-473b-bd2b-8862ddadad8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:11:45.668719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 763/1510] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:11:45 2022] DELETE /identity/v3/projects/5f88f92a6f9f421d949e8c769b637b16 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:11:54.977569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=62 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:11:55.664402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:13:26.257737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:26.258630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:26.258630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:26.260682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:13:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:26.266681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a81f-3f74-424a-8488-07ba43f25209 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:26.267149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a81f-3f74-424a-8488-07ba43f25209 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:26.267253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a81f-3f74-424a-8488-07ba43f25209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:26.358983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7a47a81f-3f74-424a-8488-07ba43f25209 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:26.360257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7a47a81f-3f74-424a-8488-07ba43f25209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:26.473713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:26.630946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:26.633413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:26.762321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd834e-c90a-4a8d-b5f1-9a3f770b458a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:26.768642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:26.769317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:26.769604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:26.769885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:26.779432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:26.781155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93070b-1898-4357-bb0e-41e121b83aab service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:26.791239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 749/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:27.616360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce474-2ce6-4902-8421-09e40320fa51 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:27.616990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce474-2ce6-4902-8421-09e40320fa51 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:27.616990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ce474-2ce6-4902-8421-09e40320fa51 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:27.618350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:27.624307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961507-de7d-485c-b758-fc6a6fb6ccf2 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:27.624986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961507-de7d-485c-b758-fc6a6fb6ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:27.625090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961507-de7d-485c-b758-fc6a6fb6ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:27.646692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2f961507-de7d-485c-b758-fc6a6fb6ccf2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:27.647946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2f961507-de7d-485c-b758-fc6a6fb6ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:27.657333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:27.817648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce474-2ce6-4902-8421-09e40320fa51 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:27.843914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ce474-2ce6-4902-8421-09e40320fa51 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:27.850111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:27.850772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:27.851059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:27.851340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:27.860011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:27.861668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17399cb-9749-40e5-aa79-37e6e5ca7a09 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:27.871441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 766/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:28.068470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0ef2-9739-43a7-955a-00cc7200b948 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:28.068736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0ef2-9739-43a7-955a-00cc7200b948 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:28.069011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0ef2-9739-43a7-955a-00cc7200b948 None None] PATH_INFO: `/v3` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:28.070372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 751/1517] 10.222.0.48 () {54 vars in 925 bytes} [Thu Jul 21 12:13:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:13:28.077966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980cf3-5eee-4f4a-b45f-eaa0f87c8492 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:28.078260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980cf3-5eee-4f4a-b45f-eaa0f87c8492 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:28.078532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980cf3-5eee-4f4a-b45f-eaa0f87c8492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:28.141090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-47980cf3-5eee-4f4a-b45f-eaa0f87c8492 None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:28.142239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-47980cf3-5eee-4f4a-b45f-eaa0f87c8492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:28.161060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:13:28.221974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 767/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 21 12:13:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:37.641330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=56 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:13:37.725164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c14c89-99e1-48a4-b781-8f4b37529403 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:37.725362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c14c89-99e1-48a4-b781-8f4b37529403 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:37.725613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c14c89-99e1-48a4-b781-8f4b37529403 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:37.727041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 752/1519] 10.222.0.18 () {54 vars in 961 bytes} [Thu Jul 21 12:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:37.733522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc29e-6cc0-4094-b37d-bb11f8a360fd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:37.734292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc29e-6cc0-4094-b37d-bb11f8a360fd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:37.734463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc29e-6cc0-4094-b37d-bb11f8a360fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:37.757099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-942cc29e-6cc0-4094-b37d-bb11f8a360fd None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:37.758357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-942cc29e-6cc0-4094-b37d-bb11f8a360fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:37.768402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 768/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Jul 21 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:37.779404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c14c89-99e1-48a4-b781-8f4b37529403 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:37.800103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c14c89-99e1-48a4-b781-8f4b37529403 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:37.806145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:37.808073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:37.808364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:37.808635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:37.817611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:37.819206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11998a72-2436-4aea-97e4-ad8bc28268c3 service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:37.830036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 753/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:39.136356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf743e-26f1-4d83-abf1-a486afac8aa7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:39.136888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf743e-26f1-4d83-abf1-a486afac8aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:39.136888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf743e-26f1-4d83-abf1-a486afac8aa7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:39.138283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:39.145716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a40298-004a-47f1-baa5-452ffe78c9df None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:39.146163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a40298-004a-47f1-baa5-452ffe78c9df None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:39.146323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a40298-004a-47f1-baa5-452ffe78c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:39.168793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-56a40298-004a-47f1-baa5-452ffe78c9df None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:39.170030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-56a40298-004a-47f1-baa5-452ffe78c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:39.179727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:39.340668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf743e-26f1-4d83-abf1-a486afac8aa7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:39.357781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf743e-26f1-4d83-abf1-a486afac8aa7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:39.364903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:39.365550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:39.365828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:39.366106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:39.375312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:39.377022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a3a9-6575-4ffe-b7df-9c1c4a22f597 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:39.387715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 770/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:40.221248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c275b-93df-4065-8307-cf315b0f909f None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:40.221817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c275b-93df-4065-8307-cf315b0f909f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:40.221817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c275b-93df-4065-8307-cf315b0f909f None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:40.223273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 12:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:40.228679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e8e85-b80a-436f-a7e4-bbc6d99884dd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:40.229170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e8e85-b80a-436f-a7e4-bbc6d99884dd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:40.229272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e8e85-b80a-436f-a7e4-bbc6d99884dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:40.372517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d69e8e85-b80a-436f-a7e4-bbc6d99884dd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:40.373759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d69e8e85-b80a-436f-a7e4-bbc6d99884dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:40.382638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:40.537837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c275b-93df-4065-8307-cf315b0f909f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:40.554067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c275b-93df-4065-8307-cf315b0f909f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:40.560722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:40.561378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:40.561658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:40.561955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:40.571420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:40.573067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2866c-d013-4a48-ac1b-e8610684d1c9 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:40.582948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 756/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:44.593039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:44.593751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:44.593751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:44.595664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:13:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:44.601454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc436-9a7e-41bf-97af-bad80489a51c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:44.601920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc436-9a7e-41bf-97af-bad80489a51c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:44.602024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cc436-9a7e-41bf-97af-bad80489a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:44.623193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2c3cc436-9a7e-41bf-97af-bad80489a51c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:44.624502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2c3cc436-9a7e-41bf-97af-bad80489a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:44.633181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:44.718657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:44.720844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:44.813563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271ae1-4510-4ea9-ade4-3c84157b3311 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:44.817788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:44.818438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:44.818728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:44.819004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:44.829207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:44.830657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b33cc7-86bb-4a86-8ee3-da182f9452ce service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:44.841085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 773/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:46.879949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:46.880527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:46.880527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:46.881897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:46.887707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63a50-3d93-42a4-9e61-6991c0d70a1c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:46.888168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63a50-3d93-42a4-9e61-6991c0d70a1c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:46.888315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63a50-3d93-42a4-9e61-6991c0d70a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:46.906499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-78b63a50-3d93-42a4-9e61-6991c0d70a1c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:46.907748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-78b63a50-3d93-42a4-9e61-6991c0d70a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:46.916035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:46.953183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:46.955308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:46.970604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9eb8a-b459-4f27-8ac3-3a6bb07be5fe None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:46.975072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:46.975758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:46.976039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:46.976334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:46.985554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:46.986978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5af08-15db-408c-b5b0-9250bdc6ea43 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:46.996624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 759/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:50.583251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:50.583848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:50.583848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:50.585106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:50.591680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3145-1f4d-488e-a8a2-266fd11e47bf None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:50.592471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3145-1f4d-488e-a8a2-266fd11e47bf None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:50.592629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3145-1f4d-488e-a8a2-266fd11e47bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:50.619863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ee0b3145-1f4d-488e-a8a2-266fd11e47bf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:50.621086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ee0b3145-1f4d-488e-a8a2-266fd11e47bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:50.629933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:50.936457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:50.939290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:51.025664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:13:51.041550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6ecbb-49ae-4d2a-a810-e13e4889793a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:51.045597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:51.046256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:51.046533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:51.046806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:51.056468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:51.057913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a81627-ecc5-4ce7-a392-c154a24ca606 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:51.067369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 776/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:13:53.124835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:53.125356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:53.125356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:53.126903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 12:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:13:53.132819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614eb3-5cd2-4800-a734-df3808f090e6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:53.133557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614eb3-5cd2-4800-a734-df3808f090e6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:53.133735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614eb3-5cd2-4800-a734-df3808f090e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:53.156119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-30614eb3-5cd2-4800-a734-df3808f090e6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:13:53.157454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-30614eb3-5cd2-4800-a734-df3808f090e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:53.166384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:13:53.485006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:13:53.486890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:13:53.500986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a0e89-e151-44af-b05b-cc448db548f3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:13:53.504587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:13:53.505235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:13:53.505520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:13:53.505794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:13:53.515296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:13:53.516779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19ed3c-ac4b-4fdf-b0b4-f302e8ac5a89 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:13:53.526485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 762/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:14:03.149123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=11 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:14:03.522289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=33 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:24.097609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.099413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.099413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.184367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.186645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.214005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 778/1540] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:24.223064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.223968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.223968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.257928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.260172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.273773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 763/1541] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:24.279194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.297012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0555c-ae02-4f64-8bfd-d91fb7e11562 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.302198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.302883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.303152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.303415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.310349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:24.311513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.314712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.353693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 779/1542] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:24.362799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.375056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff37ca7-2912-4b7f-bce6-fd553f93eec1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.384120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.384851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.385180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.385473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.386212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.388460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ae3a-846e-4bc5-9830-2bfdb096f067 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.450762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 764/1543] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:24.455709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.485794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3180-20a9-4835-b200-7c88d4f8bb60 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.624739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.625031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e941d-bcd8-4b57-bdb9-a0329832694d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.625454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e941d-bcd8-4b57-bdb9-a0329832694d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.625622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e941d-bcd8-4b57-bdb9-a0329832694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.626974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.627773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.628530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.630052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.634370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.660447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4d0e941d-bcd8-4b57-bdb9-a0329832694d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.661794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4d0e941d-bcd8-4b57-bdb9-a0329832694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.676544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 765/1544] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:24.684861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903ec0-ab65-4bd3-be19-67ae127e2657 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.685244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903ec0-ab65-4bd3-be19-67ae127e2657 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.685550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903ec0-ab65-4bd3-be19-67ae127e2657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.689004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 780/1545] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:24.694249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.723146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7a903ec0-ab65-4bd3-be19-67ae127e2657 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.724403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7a903ec0-ab65-4bd3-be19-67ae127e2657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.725099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51d12f-6aef-4e69-8992-922c415ddc8f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.738418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 766/1546] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:24.739898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77bc3-a216-46ae-bbe0-0fcadcf121a6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.740250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77bc3-a216-46ae-bbe0-0fcadcf121a6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.740493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa77bc3-a216-46ae-bbe0-0fcadcf121a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.750123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.752491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.753366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.754165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.755899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.759304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.773513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7aa77bc3-a216-46ae-bbe0-0fcadcf121a6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.774780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7aa77bc3-a216-46ae-bbe0-0fcadcf121a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.788986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 767/1547] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:24.791330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.791719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.791993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.801076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 781/1548] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:15:24.809623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.840730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f38ff0-f955-4f7e-a34a-51e3c678a579 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.849064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:24.850651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:24.864685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 768/1549] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:24.866268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.873293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.874923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.876033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.876792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.878535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.881996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.887700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fc1dd-9ee1-498d-a8e9-8a354293f64f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.890524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.891193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.891493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.891768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.892491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:24.892796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.894829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:24.894829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:15:24.894829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:15:24.907532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.921726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 782/1550] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:24.927507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 769/1551] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:24.928191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.931898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:24.953970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2cac-4a01-4f0a-a8fa-f8409748a4d6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.959136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c15b13-b9bc-4a55-a91a-88c32629e68b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:24.959351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.959682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.959963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.960240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] PATH_INFO: `/v3/projects/f3726ac8b83847468671bd930deb5492/users/a14ddfab91e94f6db6d7f6596ca31916/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.977929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:24.979587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:24.980411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:24.981776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:24.983614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:24.984498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.991178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3726ac8b83847468671bd930deb5492, user_id=a14ddfab91e94f6db6d7f6596ca31916, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:24.991785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:24.993290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.032022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 783/1552] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.034269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.037971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 770/1553] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:24 2022] PUT /identity/v3/projects/f3726ac8b83847468671bd930deb5492/users/a14ddfab91e94f6db6d7f6596ca31916/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:15:25.044043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.057674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dca8-dc7b-4823-a1a1-7180c2e09f6d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.180045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.180795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.180795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.181039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.181673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.182724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:25.182724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:15:25.182724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:15:25.183958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d0fe-91c7-46d6-895b-a7a85941381b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.235631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe759f-868f-4aae-aa8b-d2bb6d2896c9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.241804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.242505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.242940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.243271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.244500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.247306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b945ef-72e8-4a94-8681-427b9b201126 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.260385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 784/1554] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 365 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.263436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732633d9-7abf-4268-aaf8-58ab7196fed7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.264382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732633d9-7abf-4268-aaf8-58ab7196fed7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.265335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732633d9-7abf-4268-aaf8-58ab7196fed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.280937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 771/1555] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.283532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c98e6-1ab8-42b4-a6ea-9ff5fffe4914 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.283532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c98e6-1ab8-42b4-a6ea-9ff5fffe4914 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.283532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c98e6-1ab8-42b4-a6ea-9ff5fffe4914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.292622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-732633d9-7abf-4268-aaf8-58ab7196fed7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.294009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-732633d9-7abf-4268-aaf8-58ab7196fed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.303797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-623c98e6-1ab8-42b4-a6ea-9ff5fffe4914 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.305035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-623c98e6-1ab8-42b4-a6ea-9ff5fffe4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.309432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:25.354818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:25.366043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 785/1556] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.367907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bd9dc-4c69-4e80-b522-0328d39321da None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.368360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bd9dc-4c69-4e80-b522-0328d39321da None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.368638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bd9dc-4c69-4e80-b522-0328d39321da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.389870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 772/1557] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.389870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.389870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.389870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.398004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-277bd9dc-4c69-4e80-b522-0328d39321da None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.399275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-277bd9dc-4c69-4e80-b522-0328d39321da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.405640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.406844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.408885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 786/1558] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.413811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14802-3c0b-4951-a44d-fa9285b78691 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.414417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14802-3c0b-4951-a44d-fa9285b78691 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.414919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14802-3c0b-4951-a44d-fa9285b78691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.419391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 773/1559] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.425205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.435799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5cc14802-3c0b-4951-a44d-fa9285b78691 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.437686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5cc14802-3c0b-4951-a44d-fa9285b78691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.437818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e239c90-9a60-4d7f-9ebc-3e61ea547eb5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.445274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.446395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.446842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.447306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.448541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.450267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:25.450267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:15:25.450267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:15:25.450914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 787/1560] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.452708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.453199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.453650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.454101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.478264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.479507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.481324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 774/1561] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/users => generated 283 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.482729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.492178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 788/1562] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.493069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.495922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ee6de-2751-4bf5-b6ae-4769307f3a29 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.502247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.503234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.503672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.504078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.504899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.507418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e03-7f92-4fb0-893a-d9425c1d6d2f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.535212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 775/1563] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.537201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4199ab-3211-474d-b196-1c97126a61f1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.537483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4199ab-3211-474d-b196-1c97126a61f1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.537759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4199ab-3211-474d-b196-1c97126a61f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.556690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0b4199ab-3211-474d-b196-1c97126a61f1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.558101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0b4199ab-3211-474d-b196-1c97126a61f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.564965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42991-ee95-4158-ba79-6a455f8cc065 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.570010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 776/1564] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.570406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.571297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.571689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.571960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.572229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4e12c-0e1b-4e55-b0f5-20e1ca2c5c00 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.572725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4e12c-0e1b-4e55-b0f5-20e1ca2c5c00 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.573198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4e12c-0e1b-4e55-b0f5-20e1ca2c5c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.587941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.589532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8d85b-8951-43c0-8828-ec3e36c24fb5 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.597726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a6f4e12c-0e1b-4e55-b0f5-20e1ca2c5c00 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.599876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a6f4e12c-0e1b-4e55-b0f5-20e1ca2c5c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.603168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 789/1565] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:25.605244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4227-eda4-4f13-9f53-948446674750 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.605718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4227-eda4-4f13-9f53-948446674750 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.606184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c4227-eda4-4f13-9f53-948446674750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.617607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 777/1566] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.617607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d24cf5-80ee-4628-9110-bcec47553018 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.617607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d24cf5-80ee-4628-9110-bcec47553018 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.617607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d24cf5-80ee-4628-9110-bcec47553018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.634066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-081c4227-eda4-4f13-9f53-948446674750 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.635555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-081c4227-eda4-4f13-9f53-948446674750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.644081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-72d24cf5-80ee-4628-9110-bcec47553018 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.645385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-72d24cf5-80ee-4628-9110-bcec47553018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.646757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 790/1567] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.647699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c4227-eda4-4f13-9f53-948446674750 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.658882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 778/1568] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.660441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.660743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.661302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.667307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c4227-eda4-4f13-9f53-948446674750 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.676497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.678364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.678935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.679753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.680912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.681788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:25.683057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:25.684230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.692960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 779/1569] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:25.694876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.708559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 791/1570] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:25.708559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.713319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be6b48-9fb6-42bc-b87e-851e58d0edf2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.719698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.720799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.721332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.721964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.723198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.724900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80ef2-a6d7-4d3f-8c6c-af4c029e9877 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.725432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.731581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.732399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.732861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.733328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.734084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:25.734393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.736152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.738986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 780/1571] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:25.742804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.751275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 792/1572] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.752543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.754738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f72a94-fbc5-4339-83be-1eeaa9fce5e7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.762662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.763680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.763977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.764256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.765013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:25.765531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.767241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.773828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782601-7234-4d08-9e38-cf1e39246d84 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.781047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.781737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.782047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.782297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.783040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:25.783345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.785090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.787782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 781/1573] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.790430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.807583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 793/1574] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.807583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.811375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd6c58-941a-4478-b9bb-719a872a2b40 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.819647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1007f-2eaf-4368-a977-e19827fc7c46 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.820022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.821115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.821607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.822080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.823432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:25.824009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.826165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.826984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.827341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.827446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.827856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.828947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:25.829081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.830937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.849190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 794/1575] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.850218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.852228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 782/1576] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:25.853200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.869335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeddb8-09b9-457e-8a80-a2956a8c029c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.879040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f019-6d5c-4401-a852-0f99aa6a1c38 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.882756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.884102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.884724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.885085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.885415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.885853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.886272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.886537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.886728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.887342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.889155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.890859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.904109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 783/1577] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:25.905862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 795/1578] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:25.907065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.909359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.923066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ce25d-8358-48d9-9570-b5bb8a00b293 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.930239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01d7c8-1f8e-4cab-a701-b62239ee7ba4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:25.930428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.931341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.931840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.932369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.934023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.938271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.954274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:25.955680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:25.956219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:25.956921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:25.958123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:25.960484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:25.986893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 784/1579] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.988394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:25.994824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 796/1580] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:25.995758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.011402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:26.091406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ce940-ea79-4fa6-b94d-854f0c8f6ca8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.104975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570fda6-81c1-432c-811c-3184b3c3742b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.111891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.113649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.114429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.115233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.117239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.118399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.119105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.119389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.119703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.120419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.122527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.123776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.144259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:26.180532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 785/1581] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.184090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 797/1582] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:25 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.186849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.187222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.288958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b02ab-c03d-4195-a431-ade756f83b15 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.296957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.298134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.298674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.299057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.300675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.304431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.305833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9983a0-b729-4361-aa78-d329a876d6b2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.313645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.315101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.315605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.316060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] PATH_INFO: `/v3/projects/41b8b45e25af4c498afb80b94362fe95/users/c3077763acd8487291f7005bd1da77c2/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.340738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b8b45e25af4c498afb80b94362fe95, user_id=c3077763acd8487291f7005bd1da77c2, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.345147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.361151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 798/1583] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.363774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.367272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 786/1584] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:26 2022] PUT /identity/v3/projects/41b8b45e25af4c498afb80b94362fe95/users/c3077763acd8487291f7005bd1da77c2/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:26.373209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.481199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b541850-579a-4954-a287-4900ffc41728 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.482229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d233b7-1deb-4731-9df8-e1be219a81a3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.489450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.491007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.491158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.491531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.491989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] PATH_INFO: `/v3/projects/58473544a42d46eca9a07c059dff2fc7/users/2e6a837b493b47479bf6e5a43a498d96/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.492366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.492808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.493765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.495181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.500822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.520101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=58473544a42d46eca9a07c059dff2fc7, user_id=2e6a837b493b47479bf6e5a43a498d96, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.521324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:26.521324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:15:26.521324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:15:26.523668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.541201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 787/1585] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.542602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.552748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 799/1586] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:26 2022] PUT /identity/v3/projects/58473544a42d46eca9a07c059dff2fc7/users/2e6a837b493b47479bf6e5a43a498d96/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:26.553984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.569377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d2484-a2b5-4741-8ca6-4a0fcd5f6096 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.579065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:26.682588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5004-b0d3-4b3f-bf07-a1d28096c9fd admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.697148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.698497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.698967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.699453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.700941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.706127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.706587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.708515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.709507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.710386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.713272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.718648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.739517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:26.753460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 800/1587] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.754705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.767794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 788/1588] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/users => generated 303 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.769391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.776355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e11b-5296-4d8e-aebe-ab1725e33b36 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.786929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.788656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.789217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.789829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.791579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.794599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39a066-092f-4e6b-8147-f5fc521dad43 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.795905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29af30-f05f-4787-92fe-ddc6fe3d8199 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.801703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.802662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.803055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.803473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.804645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.808419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.838629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 801/1589] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/users => generated 293 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.840909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dcf96-a012-4885-8491-fe11ef145847 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.841539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dcf96-a012-4885-8491-fe11ef145847 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.842015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dcf96-a012-4885-8491-fe11ef145847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.855171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 789/1590] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/users => generated 293 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:26.857011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.873690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7d7dcf96-a012-4885-8491-fe11ef145847 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:26.875216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7d7dcf96-a012-4885-8491-fe11ef145847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:26.876767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491444-87cf-45ac-b871-d51566bd6964 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.885921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.887183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.887808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.888400 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.889940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.895902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:26.903908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 802/1591] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:26.906259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.906707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.907165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.922417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 790/1592] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:26.923981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:26.949563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d5972-867b-401b-b33b-0ea1020fbfb3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:26.952178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] MFA Rules not processed for user `2e6a837b493b47479bf6e5a43a498d96`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:26.954274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:26.961717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:26.963141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:26.963738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:26.965653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:26.967344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:26.973968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.005663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 791/1593] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.007889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.028458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf0c1-0a4c-499e-b83c-abef8ba1b264 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.034147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.035126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.035633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.036355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.037416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.039587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.054794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 792/1594] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.056041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.068386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b87cd-f11f-4fa3-95d5-f95320cc946b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.074890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.076237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.077435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.077906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.078825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.080817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.096265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 793/1595] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.097774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.098514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 803/1596] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:27.101144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.113045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bda50d-111c-4836-9e3a-1fb7f9067df4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.122837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.124017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.124518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.125350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.126377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.128399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.147599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 794/1597] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.151039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.188123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e044-7532-45d9-933d-b1fb3e5016df admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.197051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.198433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.198989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.199634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.201602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.208249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.254749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf58ad-ce86-4d1e-a053-dfa0796f0bd8 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.256583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 795/1598] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.259040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.261991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.263231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.263781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.264279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.283313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.283884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b8aa-c544-4330-bc48-de73bf274f31 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.285677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.295946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.297922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.298716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.299357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.301698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.305152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.306368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 804/1599] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:27.307586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.318416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:27.327708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b5f7-47ae-4f84-9e44-9edc45ab2a9b service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.340670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 796/1600] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.340670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.341752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.341752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.341752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.342206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.343280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.346534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.363467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8e3c7-7a16-4fae-a0a2-e9df44ea7b16 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.372700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 805/1601] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.372700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.374297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.375633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.376192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.376709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.378394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.382909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.389037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a827-10e9-48fc-8e9e-749a101e9cc1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.399265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.403088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.403088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.404069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.405339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.415159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 797/1602] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.416465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.420633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.445644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc85ae0-7443-4ddb-b2cb-61997596ced4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.455442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.457317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.457885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.458510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.460104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.464095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.484363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 806/1603] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:27 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:27.486628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.492348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 798/1604] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:27.494645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.530688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeac094-8b58-4954-81cc-bd611f9cfef3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.663214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fb2ff-0eea-4e19-bc45-651c5abf0cd1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.666829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.667794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.668079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.671632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] PATH_INFO: `/v3/projects/2409e2644eb14d6b9545088c08d22273/users/b3fe0e12c72c48fcb9ce277ebaa910f3/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.675003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.677585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.677950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.678394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] PATH_INFO: `/v3/projects/0a35c9da3fef4da09bad1a4daf6c0cbb/users/5ca9b25822654c55a3882e8ab2a10a96/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:27.698847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2409e2644eb14d6b9545088c08d22273, user_id=b3fe0e12c72c48fcb9ce277ebaa910f3, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.707118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.710884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a35c9da3fef4da09bad1a4daf6c0cbb, user_id=5ca9b25822654c55a3882e8ab2a10a96, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:27.721571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:27.759795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 799/1605] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:27 2022] PUT /identity/v3/projects/2409e2644eb14d6b9545088c08d22273/users/b3fe0e12c72c48fcb9ce277ebaa910f3/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:27.759795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.772380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 807/1606] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:27 2022] PUT /identity/v3/projects/0a35c9da3fef4da09bad1a4daf6c0cbb/users/5ca9b25822654c55a3882e8ab2a10a96/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:27.773950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:27.821575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:27.906722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88d34e-8740-4847-b117-b8730a114769 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.980740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973c476-6a17-42d1-a1b5-be5ee4aedfbe admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:27.993226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:27.995178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:27.995980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:27.996618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] PATH_INFO: `/v3/projects/c7d349f62c0c4747bd1748ca0a4bde00/users/f705392900ef4f158b3bbc6f910fd67b/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.025402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:28.026889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.027358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.027800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] PATH_INFO: `/v3/projects/31221806b70d4f9a80fbd83bfa1f371e/users/90bf7de26b6740afaf4addc6ed1c7f98/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.034952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d349f62c0c4747bd1748ca0a4bde00, user_id=f705392900ef4f158b3bbc6f910fd67b, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:28.038970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:28.071033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31221806b70d4f9a80fbd83bfa1f371e, user_id=90bf7de26b6740afaf4addc6ed1c7f98, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:28.076601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:28.079538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 808/1607] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:27 2022] PUT /identity/v3/projects/c7d349f62c0c4747bd1748ca0a4bde00/users/f705392900ef4f158b3bbc6f910fd67b/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:28.084057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:28.123322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 800/1608] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:27 2022] PUT /identity/v3/projects/31221806b70d4f9a80fbd83bfa1f371e/users/90bf7de26b6740afaf4addc6ed1c7f98/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:28.124980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:28.225284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:28.290691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e8350-3dd9-4071-81a1-ec62b47a7f24 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:28.352820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4132d8-727d-420b-b093-bf3acd901ed7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:28.358881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:28.360284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.360833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.361278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] PATH_INFO: `/v3/projects/eb96ce762266425bb8b8dea100ad98bf/users/8b56666bfe46476ba30f5129aefbacee/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.386569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb96ce762266425bb8b8dea100ad98bf, user_id=8b56666bfe46476ba30f5129aefbacee, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:28.406327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133ae47-8737-4196-aaca-5e94978e71fd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:28.437720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:28.439047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.439571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.440019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.441512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:28.444437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 801/1609] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:28 2022] PUT /identity/v3/projects/eb96ce762266425bb8b8dea100ad98bf/users/8b56666bfe46476ba30f5129aefbacee/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:28.446779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a345-72fe-4b19-99cd-d4b3a846a6dd None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.447651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a345-72fe-4b19-99cd-d4b3a846a6dd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.448190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a345-72fe-4b19-99cd-d4b3a846a6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.449601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad406ce-17da-467e-8db5-5e29c20f88fd admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:28.498566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9023a345-72fe-4b19-99cd-d4b3a846a6dd None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:28.500242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9023a345-72fe-4b19-99cd-d4b3a846a6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:28.618495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 809/1610] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/users => generated 289 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:28.618495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.618495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.618495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.624465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:28.624465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 802/1611] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:28.624465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2230-823e-466a-8d97-0efffa85c2f3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.624465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2230-823e-466a-8d97-0efffa85c2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.624465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd2230-823e-466a-8d97-0efffa85c2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.641430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] MFA Rules not processed for user `5ca9b25822654c55a3882e8ab2a10a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:28.641820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f2bd2230-823e-466a-8d97-0efffa85c2f3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:28.644306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f2bd2230-823e-466a-8d97-0efffa85c2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:28.644584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:28.666393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 803/1612] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:28.669508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f06e-6c33-4209-bc78-14d02b549f84 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.677277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f06e-6c33-4209-bc78-14d02b549f84 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.678181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51f06e-6c33-4209-bc78-14d02b549f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.720725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5a51f06e-6c33-4209-bc78-14d02b549f84 None None] MFA Rules not processed for user `90bf7de26b6740afaf4addc6ed1c7f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:28.723446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5a51f06e-6c33-4209-bc78-14d02b549f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:28.883651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 804/1613] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:28.884142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.884656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.885147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.912720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 810/1614] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:28.912720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:28.929443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a6ac5-a287-4bf2-88f6-1fbcc40ecf29 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:28.945829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:28.946975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:28.947451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:28.948944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] MFA Rules not processed for user `8b56666bfe46476ba30f5129aefbacee`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:28.949152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:28.950265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:28.951130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:28.954379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:28.983897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 811/1615] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:28.985700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:29.170257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 805/1616] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:29.173207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:29.216778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b489-7acc-48e5-8a48-78a231a41d9d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:29.231296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:29.234420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.236343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.237868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.305687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:29.319838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:29.342940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:29.385473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 812/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:29.390054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:29.424873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b76c19-4e0f-48b1-b923-1ad6229843b8 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:29.446652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:29.447904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.448367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.449790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.451353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:29.453398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:29.463300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5b033-fb50-4acd-81fc-4afc67864693 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:29.463852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cddc5-ce20-4b00-87cd-7ecedec2a793 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:29.471122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:29.472258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.472708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.473180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.524750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:29.527394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9b485-a198-4e4a-9b5c-8edb655146e2 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:29.581072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 806/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:29.585839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f2cd-9ab0-4a95-97b3-8dcc000f2299 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.585839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f2cd-9ab0-4a95-97b3-8dcc000f2299 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.587271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f2cd-9ab0-4a95-97b3-8dcc000f2299 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.591115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 807/1619] 10.222.0.18 () {54 vars in 962 bytes} [Thu Jul 21 12:15:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:29.592597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 813/1620] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:29 2022] POST /identity/v3/projects => generated 403 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:29.598562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f2cd-9ab0-4a95-97b3-8dcc000f2299 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:29.600096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeeb7-1653-4594-9897-b24f39358522 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.600620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeeb7-1653-4594-9897-b24f39358522 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.601057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eeeb7-1653-4594-9897-b24f39358522 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.607261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 814/1621] 10.222.0.18 () {54 vars in 962 bytes} [Thu Jul 21 12:15:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:29.608733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eeeb7-1653-4594-9897-b24f39358522 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:29.796299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f2cd-9ab0-4a95-97b3-8dcc000f2299 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:29.812482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:29.814775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.815455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.817368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] PATH_INFO: `/v3/projects/dc78cbf09ae14a08961bf1b2089d851b/users/0807af84992244c9b19318346c95d2ac/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.821404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eeeb7-1653-4594-9897-b24f39358522 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:29.829648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:29.830358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.830845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.831798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.831978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:29.834118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c1681-e6e2-4f7b-873c-329297501ff5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:29.882252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 815/1622] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:29 2022] POST /identity/v3/users => generated 308 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:29.887498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.887755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.888084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:29.906242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc78cbf09ae14a08961bf1b2089d851b, user_id=0807af84992244c9b19318346c95d2ac, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:29.928462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fda8b0-aa13-4e7d-90f8-283d28f390b1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:29.975509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 808/1623] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:29 2022] PUT /identity/v3/projects/dc78cbf09ae14a08961bf1b2089d851b/users/0807af84992244c9b19318346c95d2ac/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:29.982305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:29.984835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:29.986362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.025328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:30.027302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:30.035783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:30.039434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:30.226680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 809/1624] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Jul 21 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:30.229567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.287871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 816/1625] 10.222.0.18 () {58 vars in 1059 bytes} [Thu Jul 21 12:15:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:30.289455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.390000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9854e5-aa7d-4b54-b2fa-07f073c35634 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.399772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.401073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.401646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.403133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.405092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.411122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.437164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:30.451151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 810/1626] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:30.453135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.474249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:30.517269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a73079-44c6-4829-913d-88cfe7d3b755 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.532931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.534306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.534942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.535799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.545103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd437257-aa3e-4dc3-8f29-63b26f7789fd None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.562648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.564131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.564403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.564815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.565588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.568294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.623019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 817/1627] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:30.624033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.649901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9a84-7fca-457f-a13d-98c68ae7be2f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.657515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.657766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.657867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.658106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.666635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.668764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.681535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 811/1628] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:30.701851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.737191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4c983-069f-46c9-9d67-55310d50fbac service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.751456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.752380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.753508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.753747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.754135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.758501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] PATH_INFO: `/v3/projects/858fc715311d485484c1c24e721f4b21/users/b889ee4de44a42dc95c94bda74cbbe3f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.770392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 818/1629] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:30.770710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:30.799759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068fac9-6d4d-4540-8efc-5e639e7eed6c service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:30.817586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:30.819446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.824918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.825691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] PATH_INFO: `/v3/projects/dc78cbf09ae14a08961bf1b2089d851b/users/0807af84992244c9b19318346c95d2ac/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.837179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc78cbf09ae14a08961bf1b2089d851b, user_id=0807af84992244c9b19318346c95d2ac, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.842598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcd7e7-d57c-419e-872a-2a27546cd46e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.844215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=858fc715311d485484c1c24e721f4b21, user_id=b889ee4de44a42dc95c94bda74cbbe3f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:30.861282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:30.899675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 819/1630] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:30 2022] PUT /identity/v3/projects/dc78cbf09ae14a08961bf1b2089d851b/users/0807af84992244c9b19318346c95d2ac/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:30.906458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:30.907263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:30.908321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:30.908494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 812/1631] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:30 2022] PUT /identity/v3/projects/858fc715311d485484c1c24e721f4b21/users/b889ee4de44a42dc95c94bda74cbbe3f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:30.913504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:31.039562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] MFA Rules not processed for user `0807af84992244c9b19318346c95d2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:31.046977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:31.227447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93b105-4b0d-47ff-b556-70aef9026e5a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:31.252510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:31.254014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.254683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.255287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.257740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:31.263074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:31.319369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 813/1632] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:31.321137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:31.368188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 820/1633] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:31.374065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012438c5-6123-42c4-90c2-9f1e6960b8cf admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:31.375970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:31.392253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:31.394388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.396216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.398284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] PATH_INFO: `/v3/projects/858fc715311d485484c1c24e721f4b21/users/b889ee4de44a42dc95c94bda74cbbe3f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.406811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] RBAC: Authorizing `identity:create_grant(project_id=858fc715311d485484c1c24e721f4b21, user_id=b889ee4de44a42dc95c94bda74cbbe3f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:31.415880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadc64e-15ae-4e4d-b1cb-e1ed75de838d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:31.470912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 814/1634] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:31 2022] PUT /identity/v3/projects/858fc715311d485484c1c24e721f4b21/users/b889ee4de44a42dc95c94bda74cbbe3f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:31.473824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eec880-c3e4-470d-83d6-392a86bf4b76 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.474442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eec880-c3e4-470d-83d6-392a86bf4b76 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.474942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eec880-c3e4-470d-83d6-392a86bf4b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.557968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:31.567630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-68eec880-c3e4-470d-83d6-392a86bf4b76 None None] MFA Rules not processed for user `b889ee4de44a42dc95c94bda74cbbe3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:31.570272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-68eec880-c3e4-470d-83d6-392a86bf4b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:31.571167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54ef2-b3f7-48aa-ba73-7a85f2bf9fc8 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:31.705384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:31.706337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.706782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.707223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.749798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:31.829469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 815/1635] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:31.829469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e09dba-c634-4ff4-8033-eadd09fdc48e None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.829469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e09dba-c634-4ff4-8033-eadd09fdc48e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.829469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e09dba-c634-4ff4-8033-eadd09fdc48e None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.833083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 816/1636] 10.222.0.18 () {54 vars in 912 bytes} [Thu Jul 21 12:15:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:31.836520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ac46-f2f1-4623-9f83-df51deca013d None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.837123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ac46-f2f1-4623-9f83-df51deca013d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.837430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94ac46-f2f1-4623-9f83-df51deca013d None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.840341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 817/1637] 10.222.0.18 () {54 vars in 912 bytes} [Thu Jul 21 12:15:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:31.842979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.843548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.844124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.853070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:31.855805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:31.868520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 821/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:31.869876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:31.880186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f69db8-55b3-4c28-b9d1-6f2690cdfbe3 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:31.882092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:31.884425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:31.885751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:31.887577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.888741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.889321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:31.905971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:31.908412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a22f4b-3074-484f-9545-bdd2d609a030 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:31.955202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 822/1639] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:31.956873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:31.957093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:31.958292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:32.017605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:32.017825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:32.140625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 818/1640] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:32.148312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:32.194630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391cd799-b430-4d61-b593-3d4585df7ec9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:32.217693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:32.221464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:32.226942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:32.229254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:32.321933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 823/1641] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:32.325566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:32.363228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5ccfb-43b2-4794-b48c-e88706db5d21 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:32.374410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:32.376917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:32.377853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:32.379045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:32.411105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:32.414696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:32.450176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:32.451954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 824/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:32.453034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:32.471842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 819/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:32.693946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:32.724319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2c31b-9f54-41e9-91d7-2f9bd87c6bab service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:32.738591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:32.740198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:32.740803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:32.741420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:32.785793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:32.789133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:32.804570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 825/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:33.733245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:34.010280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:34.041598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4bb08-5a28-4abe-9f8b-963d92ed4097 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:34.045846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe96538-7b14-47d6-a965-d8ad22ac1777 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:34.057931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:34.059148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:34.060737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:34.061619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] PATH_INFO: `/v3/users/90bf7de26b6740afaf4addc6ed1c7f98` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:34.065042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:34.065671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bf7de26b6740afaf4addc6ed1c7f98)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:34.069703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:34.070310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:34.070841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:34.073383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] PATH_INFO: `/v3/users/5ca9b25822654c55a3882e8ab2a10a96` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:34.080997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca9b25822654c55a3882e8ab2a10a96)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:34.091207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: WARNING py.warnings [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:34.091207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: warnings.warn(msg) Jul 21 12:15:34.091207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: Jul 21 12:15:34.101410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:34.115762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:34.119431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:34.280140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bf7de26b6740afaf4addc6ed1c7f98'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.330499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bf7de26b6740afaf4addc6ed1c7f98'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.438738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca9b25822654c55a3882e8ab2a10a96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.442997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bf7de26b6740afaf4addc6ed1c7f98'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.452876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bf7de26b6740afaf4addc6ed1c7f98'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.477475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca9b25822654c55a3882e8ab2a10a96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.486119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca9b25822654c55a3882e8ab2a10a96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.549982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-733d9f3d-c611-4cea-891f-7f69d43f582a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bf7de26b6740afaf4addc6ed1c7f98'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.597991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 820/1645] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:33 2022] DELETE /identity/v3/users/90bf7de26b6740afaf4addc6ed1c7f98 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:15:34.600605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03177715-3db1-4061-9764-28b484f261a8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:34.600897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03177715-3db1-4061-9764-28b484f261a8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:34.601180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03177715-3db1-4061-9764-28b484f261a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:34.636163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-03177715-3db1-4061-9764-28b484f261a8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:34.638688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-03177715-3db1-4061-9764-28b484f261a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:34.686435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca9b25822654c55a3882e8ab2a10a96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.754607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca9b25822654c55a3882e8ab2a10a96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:34.786059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 821/1646] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:34.820432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 826/1647] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:34 2022] DELETE /identity/v3/users/5ca9b25822654c55a3882e8ab2a10a96 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:34.828960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:34.830801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:34.831368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:34.866209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:34.869829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:34.881722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:34.887745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:35.157943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 822/1648] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:35.202217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:35.205726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:35.252675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695b156-d8ac-4b4f-8269-9b0cd8b5d6ac admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:35.281025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c602d9-9f8a-4eee-94ff-02c35e3e9d73 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:35.282396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:35.284187 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:35.289742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:35.310812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:35.316023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:35.317335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:35.320316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:35.320720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:35.369028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:35.376081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18693a6b-f50d-417a-9f07-bf1ad42e5403 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:35.376600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:35.382281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:35.430049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 827/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:35.442918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 823/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:37.643205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:37.645690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:37.675688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa7e78-6c49-43e4-846e-73b13ef35496 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:37.682870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:37.684473 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:37.685073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:37.685594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] PATH_INFO: `/v3/users/2e6a837b493b47479bf6e5a43a498d96` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:37.687894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6a837b493b47479bf6e5a43a498d96)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:37.690332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:37.863959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6a837b493b47479bf6e5a43a498d96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:37.903201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6a837b493b47479bf6e5a43a498d96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:37.907836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6a837b493b47479bf6e5a43a498d96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.042338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6a837b493b47479bf6e5a43a498d96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.050402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:38.090554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6a837b493b47479bf6e5a43a498d96'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.127431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 828/1651] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:37 2022] DELETE /identity/v3/users/2e6a837b493b47479bf6e5a43a498d96 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:38.133943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:38.134784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:38.134784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:38.181927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:38.184907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:38.224443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:38.280227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:38.282850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:38.374701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 824/1652] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:38.396418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:38.398098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:38.427863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309612bb-9074-4476-a04a-09a694a6e4e7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:38.444125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:38.451833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:38.452591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:38.452952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] PATH_INFO: `/v3/projects/31221806b70d4f9a80fbd83bfa1f371e` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:38.454764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31221806b70d4f9a80fbd83bfa1f371e)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:38.455998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: WARNING py.warnings [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 12:15:38.455998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: warnings.warn(msg) Jul 21 12:15:38.455998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: Jul 21 12:15:38.457731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:38.519850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440f81d-7488-4f84-92e2-5d108600f898 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:38.560437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31221806b70d4f9a80fbd83bfa1f371e', 'request_id': 'req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.573510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31221806b70d4f9a80fbd83bfa1f371e', 'request_id': 'req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.595780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31221806b70d4f9a80fbd83bfa1f371e', 'request_id': 'req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.613641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] The token cache is being invalidate because project 31221806b70d4f9a80fbd83bfa1f371e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:38.614029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.615583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.622831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 825/1653] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:38 2022] DELETE /identity/v3/projects/31221806b70d4f9a80fbd83bfa1f371e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:38.624157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:38.628637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:38.745328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582c3cc-4a0f-437b-8e23-cfcb099cf0b5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:38.749884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:38.750940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:38.751298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:38.751900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] PATH_INFO: `/v3/users/b889ee4de44a42dc95c94bda74cbbe3f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:38.753718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b889ee4de44a42dc95c94bda74cbbe3f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:38.755955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:38.808816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:38.812926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:38.813471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:38.813920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] PATH_INFO: `/v3/projects/0a35c9da3fef4da09bad1a4daf6c0cbb` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:38.821377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a35c9da3fef4da09bad1a4daf6c0cbb)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:38.831770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:38.849277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:38.851857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b889ee4de44a42dc95c94bda74cbbe3f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.873282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b889ee4de44a42dc95c94bda74cbbe3f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.906728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b889ee4de44a42dc95c94bda74cbbe3f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.907698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b889ee4de44a42dc95c94bda74cbbe3f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.931260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b889ee4de44a42dc95c94bda74cbbe3f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:38.964497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 826/1654] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:38 2022] DELETE /identity/v3/users/b889ee4de44a42dc95c94bda74cbbe3f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:38.968237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:38.971245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:39.006801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a35c9da3fef4da09bad1a4daf6c0cbb', 'request_id': 'req-d8202bc2-ecea-4135-8471-51f0b72ae467'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:39.030902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a35c9da3fef4da09bad1a4daf6c0cbb', 'request_id': 'req-d8202bc2-ecea-4135-8471-51f0b72ae467'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:39.057896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a35c9da3fef4da09bad1a4daf6c0cbb', 'request_id': 'req-d8202bc2-ecea-4135-8471-51f0b72ae467'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:39.097102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] The token cache is being invalidate because project 0a35c9da3fef4da09bad1a4daf6c0cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:39.097684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:39.101062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d8202bc2-ecea-4135-8471-51f0b72ae467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:39.105477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 829/1655] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:38 2022] DELETE /identity/v3/projects/0a35c9da3fef4da09bad1a4daf6c0cbb => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:39.111858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:39.112453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:39.112949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:39.181924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:39.187624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:39.200098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6970-1c73-4df8-b718-da5e81b5c67a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:39.218367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:39.219407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:39.219922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:39.220403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:39.229504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:39.343816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:39.421112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:39.423686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6b3a-9f57-4dfc-9964-84770f6e64e9 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:39.451319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 827/1656] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:39.508618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:39.509062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:39.509461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:39.519605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 830/1657] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:39.556463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:39.563517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:39.597985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 828/1658] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:39.602204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:39.673034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a886d25-a525-4f46-923c-1406a6c685d5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:39.702548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:39.703351 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:39.703795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:39.704171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:39.705598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:39.706113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:39.718410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:39.766626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 831/1659] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:39.769149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:39.819596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498771f-cd22-4676-8f86-94f1c770ea27 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:39.850931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:39.852668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:39.853143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:39.853626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:39.854993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:39.867406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:39.989539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 829/1660] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:39 2022] POST /identity/v3/projects => generated 357 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:39.990717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:40.004694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1513-5aa6-47c1-a242-cf231d2e462a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:40.143843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:40.146598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.146598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.146598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.147643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:40.151968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:40.190747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 832/1661] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:39 2022] POST /identity/v3/users => generated 279 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:15:40.192334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:40.206011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaafb4-22cb-4dbf-b949-26360150402f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:40.216182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:40.217502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.217502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.218783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.220530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:40.225273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:40.239468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 830/1662] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:40.245399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:40.258454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7d344-813d-4b2c-aed6-26cb4903a1c6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:40.265679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:40.266777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.267118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.267547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.268519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:40.270357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97262-7dab-4e90-8803-489f01e4a856 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:40.290529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:40.298238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 833/1663] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:40.302904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd6a55-70a9-4acd-9741-8f4c7f957a76 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:40.310302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:40.311403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.311884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.312403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] PATH_INFO: `/v3/projects/f464c5d0ff2747f3a7b53ae3e08272d6/users/c0f37cafc0ef42f6b80de5021735ae65/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.353644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f464c5d0ff2747f3a7b53ae3e08272d6, user_id=c0f37cafc0ef42f6b80de5021735ae65, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:40.358047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:40.399973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.400259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.400548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.413642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 831/1664] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:40 2022] PUT /identity/v3/projects/f464c5d0ff2747f3a7b53ae3e08272d6/users/c0f37cafc0ef42f6b80de5021735ae65/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:40.458062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] MFA Rules not processed for user `c0f37cafc0ef42f6b80de5021735ae65`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:40.459549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:40.764490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 834/1665] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:40.768870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:40.811143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:40.947156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b64947-1b55-4679-84b4-f3eb51bbf455 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:40.962959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:40.963746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:40.964031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:40.964339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:40.965078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:40.969838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:41.084284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 832/1666] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:40 2022] POST /identity/v3/projects => generated 355 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:41.084757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:41.112691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86c5d7-ece7-42d8-89ff-950cfffef40b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:41.209452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:41.331212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:41.331965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:41.332249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:41.332522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:41.333252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:41.335283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:41.397633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 835/1667] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:41 2022] POST /identity/v3/users => generated 284 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:15:41.402638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:41.449899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb75a-339c-45b3-b6fd-172e6b4fd156 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:41.472937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:41.474129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:41.474999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:41.475982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:41.480220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:41.483446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:41.580131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 833/1668] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:41.586735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:41.624168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9525884-5bfa-4757-a8f8-6e2a6d6538f7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:41.638533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:41.640826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:41.641195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:41.641415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] PATH_INFO: `/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e/users/59d32dd096574f718c1dd8808eebe348/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:41.681834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93d612c2eb7404d811f0f883c4b1b5e, user_id=59d32dd096574f718c1dd8808eebe348, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:41.689809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:41.733129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 836/1669] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:41 2022] PUT /identity/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e/users/59d32dd096574f718c1dd8808eebe348/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:41.739756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:41.778750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82871-994b-458c-8e90-bb467f1c1545 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:42.005867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:42.007114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:42.007667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:42.008218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:42.009743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:42.013707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb708-b91a-467e-b622-934754cf7968 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:42.045965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:42.046742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 834/1670] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:15:42.074589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b469-9e38-4b73-be54-321790762f78 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:42.087234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:42.090440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:42.091153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:42.091842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] PATH_INFO: `/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e/users/59d32dd096574f718c1dd8808eebe348/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:42.102148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93d612c2eb7404d811f0f883c4b1b5e, user_id=59d32dd096574f718c1dd8808eebe348, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:42.106893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:42.163989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 837/1671] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:42 2022] PUT /identity/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e/users/59d32dd096574f718c1dd8808eebe348/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:42.194727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:42.195626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:42.198538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:42.308852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] MFA Rules not processed for user `59d32dd096574f718c1dd8808eebe348`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:42.311113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:42.542379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 835/1672] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:42.558267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:42.560382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:42.580509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:42.587363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:42.621768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:42.844905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a48192-8684-44e4-8a76-51183ebef32d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:42.878291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:42.885985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:42.886471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:42.890984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:42.927427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:42.933116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:42.979242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 838/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:43.040570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93104dac-3e7e-4f14-b468-c77ce78e2819 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:43.090838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:43.096348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:43.096965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:43.100407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] PATH_INFO: `/v3/projects/858fc715311d485484c1c24e721f4b21` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:43.103009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858fc715311d485484c1c24e721f4b21)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:43.107085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:43.143004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:43.149388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:43.223146 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492731c-23dd-4fa5-92ae-37ea30725b89 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:43.235129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:43.239105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:43.240231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:43.241624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] PATH_INFO: `/v3/projects/58473544a42d46eca9a07c059dff2fc7` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:43.246244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=58473544a42d46eca9a07c059dff2fc7)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:43.251938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:43.332439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858fc715311d485484c1c24e721f4b21', 'request_id': 'req-02e223a9-a5b3-4452-a302-49d25cca1b75'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.355356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858fc715311d485484c1c24e721f4b21', 'request_id': 'req-02e223a9-a5b3-4452-a302-49d25cca1b75'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.363601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858fc715311d485484c1c24e721f4b21', 'request_id': 'req-02e223a9-a5b3-4452-a302-49d25cca1b75'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.404818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] The token cache is being invalidate because project 858fc715311d485484c1c24e721f4b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:43.407933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.414406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.421929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 836/1674] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:42 2022] DELETE /identity/v3/projects/858fc715311d485484c1c24e721f4b21 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:43.423040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:43.429182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:43.486273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58473544a42d46eca9a07c059dff2fc7', 'request_id': 'req-e034ec61-d6b0-4594-a593-51936042660c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.522707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58473544a42d46eca9a07c059dff2fc7', 'request_id': 'req-e034ec61-d6b0-4594-a593-51936042660c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.566426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58473544a42d46eca9a07c059dff2fc7', 'request_id': 'req-e034ec61-d6b0-4594-a593-51936042660c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.632268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e223a9-a5b3-4452-a302-49d25cca1b75 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:43.634986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] The token cache is being invalidate because project 58473544a42d46eca9a07c059dff2fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:43.635639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.639629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e034ec61-d6b0-4594-a593-51936042660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:43.647591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 839/1675] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:43 2022] DELETE /identity/v3/projects/58473544a42d46eca9a07c059dff2fc7 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:43.655444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bbb-ee3b-4449-a8e7-5ff5dcff398f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:43.657929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bbb-ee3b-4449-a8e7-5ff5dcff398f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:43.658577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bbb-ee3b-4449-a8e7-5ff5dcff398f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:43.700301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:43.785443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:43.789429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e4339bbb-ee3b-4449-a8e7-5ff5dcff398f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:43.792430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e4339bbb-ee3b-4449-a8e7-5ff5dcff398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:43.799946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:43.800639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:43.801077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:43.812087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:43.822255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:43.988368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:43.995213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.052149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 837/1676] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:43 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:44.053875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:44.062357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:44.123124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 840/1677] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:44.137456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.139024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.139709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.213091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:44.218602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:44.254776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 841/1678] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:44.262929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:44.306848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05487d49-9d28-4423-a97a-dcde906bc10b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:44.343727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:44.347540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.347794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.352004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.359428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:44.361713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:44.392210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.418744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 842/1679] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:44.425625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:44.455769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d93b5b-d56b-4f59-a0c9-1ba5377e88bf service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:44.456705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca8906-5409-4756-9119-306d60401fac admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:44.471930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:44.473106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.473587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.474144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.475158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:44.480261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.486495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:44.488027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.488501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.488974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] PATH_INFO: `/v3/users/0807af84992244c9b19318346c95d2ac` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.498386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0807af84992244c9b19318346c95d2ac)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:44.502157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.566304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 843/1680] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:44 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:15:44.570026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:44.798180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb55c9-8671-4422-a610-237ab2a7471d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:44.800168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0807af84992244c9b19318346c95d2ac'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:44.816073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:44.817176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.817676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.818499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.819289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0807af84992244c9b19318346c95d2ac'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:44.819670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:44.821961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.867878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0807af84992244c9b19318346c95d2ac'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:44.870007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0807af84992244c9b19318346c95d2ac'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:44.877969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 844/1681] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:44 2022] POST /identity/v3/users => generated 287 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:44.883932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:44.887347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:44.905150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0807af84992244c9b19318346c95d2ac'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:44.922287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef28837-47e0-496f-8c31-9eb2d6cbbd4b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:44.928056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 838/1682] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:44 2022] DELETE /identity/v3/users/0807af84992244c9b19318346c95d2ac => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:44.939818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:44.941434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:44.941847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:44.942231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:44.943512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:44.945441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:44.975965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 845/1683] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:15:44.989319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:45.121384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:45.140310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:45.142454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:45.167364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e3ff-66a1-4f24-bcc6-44fc848c9e03 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:45.287839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9272d0e-76f9-4aab-be82-61777d070e68 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:45.309841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:45.311576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:45.312719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:45.313210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:45.313682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:45.322531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:45.325865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:45.363648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 839/1684] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:45.370625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:45.386086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:45.387997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:45.391232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:45.391970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] PATH_INFO: `/v3/users/8b56666bfe46476ba30f5129aefbacee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:45.395601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202a7c1-fd0b-4e58-9a9c-6461f6d6498c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:45.399297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b56666bfe46476ba30f5129aefbacee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:45.405635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:45.407079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:45.407856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:45.410273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:45.410843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] PATH_INFO: `/v3/projects/4895f8bbde1c465bbc451c6122df7dd5/users/2bcb9805113f4bd280fd1a25f5e0b29d/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:45.474598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4895f8bbde1c465bbc451c6122df7dd5, user_id=2bcb9805113f4bd280fd1a25f5e0b29d, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:45.490329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4515fea-e82d-4ab3-823b-7b55015cedb3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:45.550781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 840/1685] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:45 2022] PUT /identity/v3/projects/4895f8bbde1c465bbc451c6122df7dd5/users/2bcb9805113f4bd280fd1a25f5e0b29d/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:15:45.577999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476de5-7043-46ec-8872-b98f018e264f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:45.578287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476de5-7043-46ec-8872-b98f018e264f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:45.578572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60476de5-7043-46ec-8872-b98f018e264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:45.586970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b56666bfe46476ba30f5129aefbacee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:45.620831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b56666bfe46476ba30f5129aefbacee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:45.623268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b56666bfe46476ba30f5129aefbacee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:45.663369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-60476de5-7043-46ec-8872-b98f018e264f None None] MFA Rules not processed for user `2bcb9805113f4bd280fd1a25f5e0b29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:45.670160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60476de5-7043-46ec-8872-b98f018e264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:45.700044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b56666bfe46476ba30f5129aefbacee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:45.734987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-235df516-5468-4eca-8316-f707682dce9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b56666bfe46476ba30f5129aefbacee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:45.756294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 846/1686] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:45 2022] DELETE /identity/v3/users/8b56666bfe46476ba30f5129aefbacee => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:45.793587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:45.795712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:45.797312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:45.886675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:45.892749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:45.895742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 841/1687] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:45.903469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60476de5-7043-46ec-8872-b98f018e264f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:45.907865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60476de5-7043-46ec-8872-b98f018e264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:46.237797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 847/1688] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:46.279977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60476de5-7043-46ec-8872-b98f018e264f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:46.295148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:46.298590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:46.299312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:46.299507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:46.299767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:46.300244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:46.301137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:46.309877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:46.384983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 842/1689] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:45 2022] POST /identity/v3/projects => generated 369 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:46.386610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:46.429371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:46.557930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeff3a-08cd-43dc-8ea2-40b225408c52 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:46.626251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:46.647188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7100219-49eb-4c2b-a08a-89b6078f1f73 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:46.670689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:46.671875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:46.672355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:46.672838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:46.689308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:46.692923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd3f26-6837-47ab-ba89-b31eb205b227 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:46.802391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 843/1690] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:46 2022] POST /identity/v3/users => generated 291 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:46.805351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08227fc-80dd-4945-900c-23a1997a260d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:46.805834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08227fc-80dd-4945-900c-23a1997a260d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:46.806298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08227fc-80dd-4945-900c-23a1997a260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:46.809663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:46.812940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:46.814017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:46.814806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:46.841380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:46.845475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:46.870416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 848/1691] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:46.872386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:46.913536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aba140-9749-4d16-980e-a21adac85048 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:46.920474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f08227fc-80dd-4945-900c-23a1997a260d None None] MFA Rules not processed for user `a14ddfab91e94f6db6d7f6596ca31916`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:46.922720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f08227fc-80dd-4945-900c-23a1997a260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:46.935607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:46.936360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:46.936640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:46.936916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:46.937679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:46.941355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:46.964058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 849/1692] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:46.972748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:46.997010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee96e8-3d26-408e-a64d-fd9f638bdbc8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:47.018977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:47.019961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:47.020649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:47.020934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] PATH_INFO: `/v3/projects/177290e2d0e746d09b9925538669ffdf/users/aaefd7ec3ee14782a3ddb50c71eff31a/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:47.058362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177290e2d0e746d09b9925538669ffdf, user_id=aaefd7ec3ee14782a3ddb50c71eff31a, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:47.064668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:47.109612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 850/1693] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:46 2022] PUT /identity/v3/projects/177290e2d0e746d09b9925538669ffdf/users/aaefd7ec3ee14782a3ddb50c71eff31a/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:47.110742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:47.119112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 844/1694] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:47.130672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:47.132026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:47.132026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:47.191818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] MFA Rules not processed for user `a14ddfab91e94f6db6d7f6596ca31916`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:47.193473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:47.310039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760512f-e5f8-4a01-bf62-7d00382ac0d2 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:47.329334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:47.330156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:47.330341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:47.330610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:47.331387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:47.336918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:47.380421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 851/1695] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:47.381443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:47.392243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 845/1696] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:47.407088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:47.408631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:47.430517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c048be7-d76b-4129-aa35-c3de1fe36a59 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:47.449703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:47.450689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:47.450974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:47.451253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] PATH_INFO: `/v3/projects/177290e2d0e746d09b9925538669ffdf/users/aaefd7ec3ee14782a3ddb50c71eff31a/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:47.457476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=177290e2d0e746d09b9925538669ffdf, user_id=aaefd7ec3ee14782a3ddb50c71eff31a, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:47.468777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745b2e3-19bf-4e1a-9490-3e6e56ebf71c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:47.519955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 852/1697] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:47 2022] PUT /identity/v3/projects/177290e2d0e746d09b9925538669ffdf/users/aaefd7ec3ee14782a3ddb50c71eff31a/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:47.529576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:47.529873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:47.530353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:47.550383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:47.667940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] MFA Rules not processed for user `aaefd7ec3ee14782a3ddb50c71eff31a`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:47.670172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:47.747034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b65d25-9444-4c09-b53c-7697c79ff75f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:48.033448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:48.059183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 853/1698] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:48.077526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:48.080159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:48.089863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:48.091013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:48.091565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:48.091926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:48.348489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:48.352503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b97770-c07d-46fe-a13d-e8c06d1e3b3d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:48.354922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:48.370218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:48.372878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:48.372878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:48.375994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:48.381557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 846/1699] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 975 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:48.414034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:48.418241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:48.462235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 854/1700] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:48.606488 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:48.621434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955e890-58f3-4adb-901b-c194f028c686 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:48.636538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:48.640559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:48.641091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:48.642818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:48.916069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:48.918331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f463f5b-b5db-4f39-b5fc-7242ffaa340a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:48.970258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 847/1701] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:49.386313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:49.389954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:49.695412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a661a06-614a-4170-98c4-528a74eb31db service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:49.724480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:49.726214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:49.726852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:49.731440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] PATH_INFO: `/v3/projects/dc78cbf09ae14a08961bf1b2089d851b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:49.741655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc78cbf09ae14a08961bf1b2089d851b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:49.745677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:50.034871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc78cbf09ae14a08961bf1b2089d851b', 'request_id': 'req-e01c74c6-9cde-4e6f-b19f-867de34bb32c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.057687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc78cbf09ae14a08961bf1b2089d851b', 'request_id': 'req-e01c74c6-9cde-4e6f-b19f-867de34bb32c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.083468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc78cbf09ae14a08961bf1b2089d851b', 'request_id': 'req-e01c74c6-9cde-4e6f-b19f-867de34bb32c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.112068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] The token cache is being invalidate because project dc78cbf09ae14a08961bf1b2089d851b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:50.112550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.113770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.123262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8833-9dbf-4e0d-acba-84ccabb856ae None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.123898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8833-9dbf-4e0d-acba-84ccabb856ae None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.123898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8833-9dbf-4e0d-acba-84ccabb856ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.125394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 855/1702] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:49 2022] DELETE /identity/v3/projects/dc78cbf09ae14a08961bf1b2089d851b => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:50.166199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9dbd8833-9dbf-4e0d-acba-84ccabb856ae None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:50.167664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9dbd8833-9dbf-4e0d-acba-84ccabb856ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:50.191673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:50.234153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:50.236450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:50.375854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:50.441486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 848/1703] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:50.442957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479fb2-ae44-426b-8c38-912a920ed7d6 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.443716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479fb2-ae44-426b-8c38-912a920ed7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.443966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e479fb2-ae44-426b-8c38-912a920ed7d6 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.454551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 849/1704] 10.222.0.18 () {54 vars in 912 bytes} [Thu Jul 21 12:15:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:50.467411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32bf-4e7e-44d1-a16b-c0b36731028e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.467411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32bf-4e7e-44d1-a16b-c0b36731028e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.467411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe32bf-4e7e-44d1-a16b-c0b36731028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.539147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c74c6-9cde-4e6f-b19f-867de34bb32c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:50.553022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-affe32bf-4e7e-44d1-a16b-c0b36731028e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:50.554303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-affe32bf-4e7e-44d1-a16b-c0b36731028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:50.568733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:50.569666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.569951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.570244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] PATH_INFO: `/v3/projects/eb96ce762266425bb8b8dea100ad98bf` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.573010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb96ce762266425bb8b8dea100ad98bf)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:50.575107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:50.589077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 850/1705] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:50.590527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a91ab-0c31-491a-aef0-fc0084781181 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.590852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a91ab-0c31-491a-aef0-fc0084781181 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.597435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a91ab-0c31-491a-aef0-fc0084781181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.697810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7f6a91ab-0c31-491a-aef0-fc0084781181 None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:50.699204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7f6a91ab-0c31-491a-aef0-fc0084781181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:50.724010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb96ce762266425bb8b8dea100ad98bf', 'request_id': 'req-8a0cc1c3-617a-47e9-804a-898f759e6ea6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.751745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb96ce762266425bb8b8dea100ad98bf', 'request_id': 'req-8a0cc1c3-617a-47e9-804a-898f759e6ea6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.761025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb96ce762266425bb8b8dea100ad98bf', 'request_id': 'req-8a0cc1c3-617a-47e9-804a-898f759e6ea6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.811401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] The token cache is being invalidate because project eb96ce762266425bb8b8dea100ad98bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:15:50.811736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.812532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:50.823067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 856/1706] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:15:50 2022] DELETE /identity/v3/projects/eb96ce762266425bb8b8dea100ad98bf => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:15:50.824407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:50.829566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:50.833685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 851/1707] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:50.836371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:50.836651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:50.836939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:50.901091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:50.903435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:51.093508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cc1c3-617a-47e9-804a-898f759e6ea6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.115935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:51.117294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.118128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.118656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.119614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:51.120023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 852/1708] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:51.120664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:51.120914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:51.122398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:51.124735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f217664-ef35-4aef-b723-0f99702a32d6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:51.161609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2be60-962d-4670-9324-b4ee3b9295cf None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.175407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 857/1709] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:51.180353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.180986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.181624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.182858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:51.183626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.183920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.184372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.248422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:51.256874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736d35-32ab-4862-9431-9580ada21ee5 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:51.266448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] MFA Rules not processed for user `f705392900ef4f158b3bbc6f910fd67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:51.267971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:51.306647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 853/1710] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:51.312524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.312820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.313115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.373030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:51.377286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:51.382193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:51.407762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 854/1711] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:51.413969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:51.448197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b1340-2841-4a77-9f88-23fbd5e97bc8 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.461409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:51.471458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.472050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.472769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.475166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:51.481158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:51.491657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 858/1712] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:51.492719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:51.561927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a7e3-face-40c6-b89a-8de244bc2bb6 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.571699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:51.576174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:51.576879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.577170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.577450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:51.580952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:15:51.581221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:51.585706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:51.607926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 855/1713] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:51 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:51.612395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:51.623323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:51.627127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 859/1714] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:15:51.629008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:51.658041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb79aa-c286-48fb-a585-c121fc1d5acf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.908845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edb32-f699-4d9d-9177-429ac03ab908 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:51.940174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:51.940882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:51.941173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:51.941453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.036537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.037962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.038523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.039123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.040625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.045485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.114849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 860/1715] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:51 2022] POST /identity/v3/users => generated 291 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:52.119996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.185465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539736a-a205-4e4a-8de6-663c75773cdf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.197259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.200315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.211152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.212331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.212795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.213256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.214575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.218341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.251513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 856/1716] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:52.283921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.312289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 861/1717] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:52 2022] POST /identity/v3/projects => generated 365 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:52.321133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.323408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05737021-93e6-40fd-9735-2d9ca29d4d73 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.506206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.506836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.507123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.507401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.508743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.510443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.547116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 857/1718] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:52.555924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9793-a6f2-497d-ac08-e542ad057f22 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.556199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.575770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106d086-2350-4c41-aff0-a08e7009f8ea admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.582366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.584583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.585322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.585750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.586885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.589312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.594416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.595135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.595413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.595717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.601809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.603930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.608013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 858/1719] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:52.614842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.636332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d6201-d3af-4efb-b350-2ddf6a40cac6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.643758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.645230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.645708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.646209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] PATH_INFO: `/v3/projects/b47225f861434e24ad15b59608400e75/users/feb56d2476d64c41b0fe8ca35f0fc82c/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.669481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 862/1720] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:52 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:52.675131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.682151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47225f861434e24ad15b59608400e75, user_id=feb56d2476d64c41b0fe8ca35f0fc82c, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.689290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e5bad-1861-463b-b773-266d311cc540 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.698061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:52.719271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1dc39-ff4d-4c52-aebc-fe37c59f1759 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.719820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 859/1721] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:52 2022] PUT /identity/v3/projects/b47225f861434e24ad15b59608400e75/users/feb56d2476d64c41b0fe8ca35f0fc82c/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:52.735980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.736632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.737115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.802579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:52.808003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] MFA Rules not processed for user `feb56d2476d64c41b0fe8ca35f0fc82c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:52.811990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:52.885998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.889246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.889799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.893902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.895076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.897585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.920376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 863/1722] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:52.923446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:52.945326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985431b3-5690-4cb0-a02d-afd25591bc03 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:52.956336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:52.957422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:52.957892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:52.958967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:52.960044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:52.962158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:52.986204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 864/1723] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:52.991919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:53.024035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8abc7-abab-4e51-8273-0b15805b32db admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:53.041124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:53.042990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.049358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.050065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 860/1724] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:53.050930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:53.051929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] PATH_INFO: `/v3/projects/b5ee67f639f742ee8e17275a8368bb59/users/f6a8a65704ba467086415c366b830c40/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.076588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e3d94-2ffa-482d-8d7d-a01932a6a6f3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:53.084724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:53.086099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.086906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.087764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.088810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:53.092460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:53.096694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ee67f639f742ee8e17275a8368bb59, user_id=f6a8a65704ba467086415c366b830c40, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:53.103981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3b681-1b70-4889-b785-4314ec663d97 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:53.143543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 865/1725] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:52 2022] PUT /identity/v3/projects/b5ee67f639f742ee8e17275a8368bb59/users/f6a8a65704ba467086415c366b830c40/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:53.159933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 861/1726] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:15:53 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:53.160210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf63a3-653c-4489-bdc7-ddecb891d3ac None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.160210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf63a3-653c-4489-bdc7-ddecb891d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.160210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf63a3-653c-4489-bdc7-ddecb891d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.165319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:53.274205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5ccf63a3-653c-4489-bdc7-ddecb891d3ac None None] MFA Rules not processed for user `f6a8a65704ba467086415c366b830c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:53.285038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5ccf63a3-653c-4489-bdc7-ddecb891d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:53.406444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a23c4-365a-49c9-916d-61b626325b51 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:53.428439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:53.432640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.433434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.436323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.437552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:53.445575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb2928-458c-411a-b222-7753a6835d36 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:53.484284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 866/1727] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:53.498429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.498861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.499152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.510049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 867/1728] 10.222.0.18 () {54 vars in 961 bytes} [Thu Jul 21 12:15:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:53.511289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:53.513697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:53.526119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 862/1729] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:15:53 2022] POST /identity/v3/users => generated 298 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:15:53.531030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b18-89b9-4d3c-8b65-46f059c745f6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.531785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b18-89b9-4d3c-8b65-46f059c745f6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.532597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd468b18-89b9-4d3c-8b65-46f059c745f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.596294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bd468b18-89b9-4d3c-8b65-46f059c745f6 None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:53.601890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bd468b18-89b9-4d3c-8b65-46f059c745f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:53.687315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b47afa-565c-42dc-8b22-d98c28d6c3b2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:53.704360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:53.706716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.709596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.710185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.755263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:53.760318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:53.768911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:53.803148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 868/1730] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:53.805905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:53.849401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 863/1731] 10.222.0.18 () {58 vars in 1058 bytes} [Thu Jul 21 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:53.852143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.852638 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.853134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.855012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c3b1a-7f9c-4190-bb1f-87ba861b22d8 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:53.875863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:53.879222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:53.880942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:53.884344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:53.889109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:53.896422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:53.945577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] MFA Rules not processed for user `a14ddfab91e94f6db6d7f6596ca31916`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:53.947990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:53.948224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 869/1732] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:53.962971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:54.002055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33391a-68cb-40c5-9ca2-b383252e7fa5 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:54.025189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8282-1994-4dec-b356-044638a248dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:54.027012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8282-1994-4dec-b356-044638a248dd service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:54.029071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8282-1994-4dec-b356-044638a248dd service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:54.029555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8282-1994-4dec-b356-044638a248dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:54.039133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-708c8282-1994-4dec-b356-044638a248dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:54.077166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:54.087570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c8282-1994-4dec-b356-044638a248dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:54.090036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c8282-1994-4dec-b356-044638a248dd service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:54.123033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 870/1733] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:54.125902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8282-1994-4dec-b356-044638a248dd service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:54.159382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8282-1994-4dec-b356-044638a248dd service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:54.175070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:54.179686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:54.180397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:54.180899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] PATH_INFO: `/v3/projects/a961b66ac5d742ea83aba247cd4ab243/users/6fa5ce5542634e1ba6b63bd718d41aed/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:54.201435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 864/1734] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:54.247776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a961b66ac5d742ea83aba247cd4ab243, user_id=6fa5ce5542634e1ba6b63bd718d41aed, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:54.258731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:54.319280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 871/1735] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:54 2022] PUT /identity/v3/projects/a961b66ac5d742ea83aba247cd4ab243/users/6fa5ce5542634e1ba6b63bd718d41aed/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:54.321515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:54.353771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c6fb3-053b-417a-ae93-3fd8dfcc21f9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:54.589736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:54.591923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:54.592708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:54.593249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:54.594571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:54.596631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7d390-f425-4cfb-abf4-eaba59bbe738 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:54.626021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 865/1736] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:15:54.632355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:54.654485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e8bd7-a7bd-42b5-a6af-b5fc299ec6a4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:54.674441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:54.678992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:54.683867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:54.684437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] PATH_INFO: `/v3/projects/a961b66ac5d742ea83aba247cd4ab243/users/6fa5ce5542634e1ba6b63bd718d41aed/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:54.693629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a961b66ac5d742ea83aba247cd4ab243, user_id=6fa5ce5542634e1ba6b63bd718d41aed, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:54.708959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:54.764896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871d0c-cfc3-431f-a0d9-1ec6cfa4bc05 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:54.765333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871d0c-cfc3-431f-a0d9-1ec6cfa4bc05 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:54.766635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8871d0c-cfc3-431f-a0d9-1ec6cfa4bc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:54.772270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 872/1737] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:15:54 2022] PUT /identity/v3/projects/a961b66ac5d742ea83aba247cd4ab243/users/6fa5ce5542634e1ba6b63bd718d41aed/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:54.842475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c8871d0c-cfc3-431f-a0d9-1ec6cfa4bc05 None None] MFA Rules not processed for user `6fa5ce5542634e1ba6b63bd718d41aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:54.844502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c8871d0c-cfc3-431f-a0d9-1ec6cfa4bc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:55.010635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 866/1738] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:55.027202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:55.073010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:55.073567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:55.073964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:55.158760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] MFA Rules not processed for user `c3077763acd8487291f7005bd1da77c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:55.169075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:55.194851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:55.243856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb0cdd-b540-437e-8875-514eeaf4251a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:55.256401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:55.257106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:55.262265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:55.262721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:55.290177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:55.294519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d16e-eadf-432c-8150-3d2096863857 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:55.321253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 873/1739] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:55.390157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 867/1740] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:55.409730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:55.410060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:55.410341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:55.458744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] MFA Rules not processed for user `c3077763acd8487291f7005bd1da77c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:55.466112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:55.496648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 874/1741] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:55.511967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:55.717428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:55.758230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dd65e-f977-4a48-b33d-eccf440eb4b7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:55.774432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:55.776440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:55.777174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:55.777891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:55.852496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da6aa7-ec66-4323-ad80-5e82773c1b08 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:55.860089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:55.860799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:55.861088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:55.861366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:55.884894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:55.887895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:55.944049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 868/1742] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:56.194777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:56.196649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284ff6c-276b-40fe-adc5-04846a81d6f3 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:56.309498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 875/1743] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:56.913355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:57.196929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd238-9c49-4776-bb6c-ee19fe6711b5 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:57.229213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:57.230155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:57.230444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:57.230730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] PATH_INFO: `/v3/users/feb56d2476d64c41b0fe8ca35f0fc82c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:57.233695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb56d2476d64c41b0fe8ca35f0fc82c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:57.235890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:57.343168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426293f-9f15-48bd-aed5-2a0529e23279 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:57.345375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426293f-9f15-48bd-aed5-2a0529e23279 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:57.345375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426293f-9f15-48bd-aed5-2a0529e23279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:57.375495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:57.404091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb56d2476d64c41b0fe8ca35f0fc82c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:57.420369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb56d2476d64c41b0fe8ca35f0fc82c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:57.422632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c426293f-9f15-48bd-aed5-2a0529e23279 None None] MFA Rules not processed for user `b3fe0e12c72c48fcb9ce277ebaa910f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:57.426317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c426293f-9f15-48bd-aed5-2a0529e23279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:57.453947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb56d2476d64c41b0fe8ca35f0fc82c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:57.455940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb56d2476d64c41b0fe8ca35f0fc82c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:57.487266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b6f1ff0f-a590-41aa-9dae-67d618f69c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb56d2476d64c41b0fe8ca35f0fc82c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:15:57.508662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 869/1744] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:15:56 2022] DELETE /identity/v3/users/feb56d2476d64c41b0fe8ca35f0fc82c => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:15:57.511430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:57.511846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:57.512137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:57.585094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:57.586381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:57.601344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:15:57.753219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 876/1745] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:15:57.770872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36aa7-85b6-4955-8b4c-db1c3cafc1dd None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:57.773959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36aa7-85b6-4955-8b4c-db1c3cafc1dd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:57.773959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b36aa7-85b6-4955-8b4c-db1c3cafc1dd None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:57.778552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 877/1746] 10.222.0.18 () {54 vars in 912 bytes} [Thu Jul 21 12:15:57 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:15:57.784758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:57.808205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:57.808950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:57.851316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:57.853008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:58.022687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 870/1747] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:58.062980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:58.072108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:58.103106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34345a-4cad-42a2-a683-74b3632589ee None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:58.111626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 878/1748] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:58.121007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:58.126609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:58.131138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:58.132437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:58.132804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:58.133112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:58.160941 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:58.168506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:58.204522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 871/1749] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:15:58.208548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:15:58.263295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc7a47-b6ac-4519-8e38-9106a8e50720 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:58.291643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:58.295967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:58.296478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:58.299755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:58.342471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:58.351304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca26e6-090c-4c1e-84f6-ba2e463ee5cf service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:58.380632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 872/1750] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:58.470840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c275d5-27ae-46c1-bd4a-07922889fd42 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:15:58.503988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:15:58.506084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:58.509352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:58.516996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:58.548788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:58.551654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:58.552290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:15:58.674083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] MFA Rules not processed for user `f705392900ef4f158b3bbc6f910fd67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:15:58.684869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:15:58.731723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:15:58.733705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02049312-90ec-46ec-8365-30d3737c8208 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:15:58.762359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 879/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:15:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:15:59.130984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 873/1752] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:15:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 584 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:15:59.901528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:15:59.902077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:15:59.902175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:00.074271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:00.104137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:00.107110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:00.351221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 880/1753] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:00.716658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:00.719378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:00.953870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:01.381578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0860cf-9c4a-4b0c-9c88-e8ca4d083754 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:01.411797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:01.413346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:01.416355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:01.418742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:01.470783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:01.476546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:01.579950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 874/1754] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:16:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:16:03.664138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:03.666545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:03.712138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f299e74-1099-4d0a-a545-ed95df077bf7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:03.737940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:03.739034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:03.739727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:03.740104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] PATH_INFO: `/v3/projects/b47225f861434e24ad15b59608400e75` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:03.741655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47225f861434e24ad15b59608400e75)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:03.748292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:04.030558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47225f861434e24ad15b59608400e75', 'request_id': 'req-45561536-b375-4f3a-bbb2-1a16a893b8ed'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.049711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47225f861434e24ad15b59608400e75', 'request_id': 'req-45561536-b375-4f3a-bbb2-1a16a893b8ed'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.113430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47225f861434e24ad15b59608400e75', 'request_id': 'req-45561536-b375-4f3a-bbb2-1a16a893b8ed'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.223624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] The token cache is being invalidate because project b47225f861434e24ad15b59608400e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:04.225739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.228276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.237201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 881/1755] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:03 2022] DELETE /identity/v3/projects/b47225f861434e24ad15b59608400e75 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:04.243192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:04.309620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21cf9-400a-4cbb-b2f7-1e7d3a85e925 service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:04.598776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:04.605733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:04.605733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:04.606084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] PATH_INFO: `/v3/users/6fa5ce5542634e1ba6b63bd718d41aed` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:04.607710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa5ce5542634e1ba6b63bd718d41aed)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:04.610067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:04.899528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa5ce5542634e1ba6b63bd718d41aed'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:04.971748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa5ce5542634e1ba6b63bd718d41aed'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:05.067668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa5ce5542634e1ba6b63bd718d41aed'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:05.077934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa5ce5542634e1ba6b63bd718d41aed'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:05.139280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1024b0e-9c7c-410b-aea0-1411d8dba233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa5ce5542634e1ba6b63bd718d41aed'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:05.183919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 875/1756] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:04 2022] DELETE /identity/v3/users/6fa5ce5542634e1ba6b63bd718d41aed => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:09.036017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:09.037297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:09.102528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45561536-b375-4f3a-bbb2-1a16a893b8ed admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:09.294185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cea13d-2680-4d0c-9b32-4f7b65d8ff2e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:09.295336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cea13d-2680-4d0c-9b32-4f7b65d8ff2e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:09.299843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cea13d-2680-4d0c-9b32-4f7b65d8ff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:09.429778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d3cea13d-2680-4d0c-9b32-4f7b65d8ff2e None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:09.431084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d3cea13d-2680-4d0c-9b32-4f7b65d8ff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:09.470403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:09.471346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:09.471664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:09.471957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] PATH_INFO: `/v3/projects/a961b66ac5d742ea83aba247cd4ab243` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:09.499075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=a961b66ac5d742ea83aba247cd4ab243)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:09.503547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:09.534270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:09.699090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:09.814696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 876/1757] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Jul 21 12:16:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:09.837014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a961b66ac5d742ea83aba247cd4ab243', 'request_id': 'req-4e62720e-d86d-43ce-b184-7e7572f45dde'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:09.883283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a961b66ac5d742ea83aba247cd4ab243', 'request_id': 'req-4e62720e-d86d-43ce-b184-7e7572f45dde'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:09.916526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a961b66ac5d742ea83aba247cd4ab243', 'request_id': 'req-4e62720e-d86d-43ce-b184-7e7572f45dde'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:09.980974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] The token cache is being invalidate because project a961b66ac5d742ea83aba247cd4ab243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:09.982393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:09.983724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:09.994180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 882/1758] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:09 2022] DELETE /identity/v3/projects/a961b66ac5d742ea83aba247cd4ab243 => generated 0 bytes in 955 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:10.002184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:10.002686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:10.003172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:10.053337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:10.055107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:10.154755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:10.157450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:10.325746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62720e-d86d-43ce-b184-7e7572f45dde admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:10.351641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:10.355554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:10.402562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:10.402942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:10.423623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:10.479642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:10.483104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd879b-8932-4439-b427-f8af282c9cf2 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:10.581457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 883/1759] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:10.696285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 877/1760] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 684 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:10.707410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:10.708459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:10.712276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:10.798620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:10.812849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:10.879798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 884/1761] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:10.881460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:10.914608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cf21f-2f8b-47eb-87a4-8146efc8b248 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:10.978130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:10.979828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:10.980377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:10.980717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:11.010042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:11.010854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:11.014725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11362c19-a23f-464c-b310-0602857b8fcf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:11.129905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 878/1762] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:11.132904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:11.195616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f15a90-d868-407d-8cc8-a23a1eeb8533 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:11.234360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:11.238797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:11.239676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:11.243647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:11.256418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:11.259557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:11.337742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:11.338304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:11.338861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:11.453573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 885/1763] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:11 2022] POST /identity/v3/projects => generated 371 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:11.455466 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:11.480391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] MFA Rules not processed for user `b3fe0e12c72c48fcb9ce277ebaa910f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:11.488024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:11.831000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 879/1764] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:12.050833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e95d2-5068-49e6-ad6d-d6cb0d54504a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:12.095061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:12.096682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:12.097475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:12.110043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:12.112025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:12.116301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:12.218684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:12.222020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 886/1765] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:11 2022] POST /identity/v3/users => generated 286 bytes in 767 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:12.299940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949da8e-f25a-423a-bc0e-738a609284dd None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:12.322709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:12.325500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:12.326188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:12.326630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:12.327673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:12.335185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:12.381686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 880/1766] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:12.386609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:12.435677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75729-0abf-44d9-b524-3d647a23e878 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:12.478334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:12.483696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:12.486491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:12.489512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:12.491470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:12.497852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1edd3a-7407-42f7-9b63-e13bf30c6e9e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:12.562300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:12.563224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 887/1767] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:12.585817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141bfb-1579-4459-8539-33c76a60653a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:12.612453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:12.613589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:12.613879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:12.614165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] PATH_INFO: `/v3/projects/60a80724abf045729000dc9fb8a2440e/users/85bc6b75434c4e54afcf3645a46449eb/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:12.698569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a80724abf045729000dc9fb8a2440e, user_id=85bc6b75434c4e54afcf3645a46449eb, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:12.704266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:12.763761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 881/1768] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:12 2022] PUT /identity/v3/projects/60a80724abf045729000dc9fb8a2440e/users/85bc6b75434c4e54afcf3645a46449eb/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:12.770860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:12.771182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:12.771556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:12.785379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:12.892132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:12.894599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:13.313855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 888/1769] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:13.423110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:13.535393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:13.640548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949644ae-3004-4c6b-943d-ef25ab7f0a1b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:13.671510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:13.673962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:13.677349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:13.677792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:13.711217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:13.715245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89e3b3-cb85-43a7-b634-f7d540a001af service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:13.759317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 882/1770] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:14.006915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:14.008761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:14.058063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8cc4-cd59-4e95-87e2-234f6bf7fafe None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:14.081487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:14.086280 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:14.087534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:14.088682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] PATH_INFO: `/v3/users/c0f37cafc0ef42f6b80de5021735ae65` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:14.098055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f37cafc0ef42f6b80de5021735ae65)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:14.113096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:14.451461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f37cafc0ef42f6b80de5021735ae65'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:14.497050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f37cafc0ef42f6b80de5021735ae65'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:14.502718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f37cafc0ef42f6b80de5021735ae65'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:14.574181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f37cafc0ef42f6b80de5021735ae65'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:14.619171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f37cafc0ef42f6b80de5021735ae65'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:14.656389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 889/1771] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:14 2022] DELETE /identity/v3/users/c0f37cafc0ef42f6b80de5021735ae65 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:14.658893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:14.659851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:14.660334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:14.712488 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:14.714950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:14.952160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 883/1772] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:14.989463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:14.992838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:15.242462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591b205-cf43-4cdd-8234-3d0d00ec4c52 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:15.263038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:15.267991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:15.268929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:15.270545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:15.283619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:15.287198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:15.323284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:15.333489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87c829-f85e-42b3-b9ed-6aecd4c8bcfb service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:15.334063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299cb96-9c12-433a-b06b-cb9a0275a5db None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:15.353407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:15.355032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:15.355731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:15.358785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] PATH_INFO: `/v3/users/f6a8a65704ba467086415c366b830c40` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:15.360743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a8a65704ba467086415c366b830c40)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:15.363516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 890/1773] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:15.367196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:15.491423 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a8a65704ba467086415c366b830c40'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:15.507213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a8a65704ba467086415c366b830c40'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:15.555915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a8a65704ba467086415c366b830c40'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:15.580397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a8a65704ba467086415c366b830c40'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:15.608851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a8a65704ba467086415c366b830c40'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:15.627183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 884/1774] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:15 2022] DELETE /identity/v3/users/f6a8a65704ba467086415c366b830c40 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:15.630939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12eace-eac8-4b21-bd06-75ca1ebb1475 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:15.631605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12eace-eac8-4b21-bd06-75ca1ebb1475 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:15.632226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12eace-eac8-4b21-bd06-75ca1ebb1475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:15.700143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ed12eace-eac8-4b21-bd06-75ca1ebb1475 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:15.704131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ed12eace-eac8-4b21-bd06-75ca1ebb1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:15.759985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:15.900322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 891/1775] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:15.943580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:15.950225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:16.140442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:16.250096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5dbf-d815-4072-b49c-351bffd4a54c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:16.267043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:16.268298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:16.268874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:16.269662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:16.298219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:16.300248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff9748-eda7-4c39-ba72-a02e7d9842b8 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:16.333058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 885/1776] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:18.493329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70603b-c66c-41e5-a087-b56d5454b3dd None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:18.494975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70603b-c66c-41e5-a087-b56d5454b3dd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:18.495791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70603b-c66c-41e5-a087-b56d5454b3dd None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:18.499394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 892/1777] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:16:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:16:18.600064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc55b-077d-4606-953d-d58caf26670e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:18.601094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc55b-077d-4606-953d-d58caf26670e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:18.601094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bc55b-077d-4606-953d-d58caf26670e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:18.718168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-465bc55b-077d-4606-953d-d58caf26670e None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:18.722165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-465bc55b-077d-4606-953d-d58caf26670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:19.171712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 886/1778] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:19.658041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:19.658041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:19.659237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:19.661138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 893/1779] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:16:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:16:19.698637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:19.700713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:19.701590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:19.797071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:19.800899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:19.827801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 887/1780] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:19.986261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:19.988516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:20.079661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb256-41de-4c8b-a723-e4e22862e3e0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:20.120425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:20.122761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:20.123303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:20.125115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:20.158873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:20.164151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:20.179849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 894/1781] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:20.882557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:20.884200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:20.923535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:20.925779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:20.950537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0d6e8-6ea8-4bea-9178-dd725f0a5527 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:20.958283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cc766-8761-4665-be7f-df9e6bdf9a14 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:20.978922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:20.979548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:20.982564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:20.983975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:20.987065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:20.988404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:20.991883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:20.992675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] PATH_INFO: `/v3/projects/b5ee67f639f742ee8e17275a8368bb59` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:20.995338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ee67f639f742ee8e17275a8368bb59)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.002042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.017517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.024227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.077576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 888/1782] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:21.249719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ee67f639f742ee8e17275a8368bb59', 'request_id': 'req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.286178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ee67f639f742ee8e17275a8368bb59', 'request_id': 'req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.317808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ee67f639f742ee8e17275a8368bb59', 'request_id': 'req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.331461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:21.333214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:21.360166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18a522-6ad2-47c6-bdd5-5a00e2028c63 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:21.372415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] The token cache is being invalidate because project b5ee67f639f742ee8e17275a8368bb59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:21.372926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.378550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-82d8b26d-b4ec-463a-8fac-4401fb9dbc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.385058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 895/1783] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:20 2022] DELETE /identity/v3/projects/b5ee67f639f742ee8e17275a8368bb59 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:21.387843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d376af-04bf-48ca-b670-b2db6eb260a0 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.388309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d376af-04bf-48ca-b670-b2db6eb260a0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.388770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d376af-04bf-48ca-b670-b2db6eb260a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.453850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-40d376af-04bf-48ca-b670-b2db6eb260a0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:21.455988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-40d376af-04bf-48ca-b670-b2db6eb260a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:21.520695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:21.521787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.522028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.522308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] PATH_INFO: `/v3/projects/f464c5d0ff2747f3a7b53ae3e08272d6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.524277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f464c5d0ff2747f3a7b53ae3e08272d6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.526302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.641919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f464c5d0ff2747f3a7b53ae3e08272d6', 'request_id': 'req-64516112-9222-4935-b7e6-ead4bb2f33ec'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.654956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f464c5d0ff2747f3a7b53ae3e08272d6', 'request_id': 'req-64516112-9222-4935-b7e6-ead4bb2f33ec'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.657290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:21.665026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f464c5d0ff2747f3a7b53ae3e08272d6', 'request_id': 'req-64516112-9222-4935-b7e6-ead4bb2f33ec'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.694104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] The token cache is being invalidate because project f464c5d0ff2747f3a7b53ae3e08272d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:21.696516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.700248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.707328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 889/1784] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:21 2022] DELETE /identity/v3/projects/f464c5d0ff2747f3a7b53ae3e08272d6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:21.714376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:21.715992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:21.724972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 896/1785] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:21.735416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.736406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.737120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.798470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:21.801623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:21.809953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64516112-9222-4935-b7e6-ead4bb2f33ec admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:21.815671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:21.816730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.817024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.817310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] PATH_INFO: `/v3/users/59d32dd096574f718c1dd8808eebe348` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.819148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d32dd096574f718c1dd8808eebe348)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.820945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 897/1786] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:21.824764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.826389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:21.849776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b1551-084b-4e70-9be1-b58250b15a2d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:21.863053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:21.865186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.865672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.866519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.869528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:21.871046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.875095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.912299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 898/1787] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:16:21.916573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:21.930248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d32dd096574f718c1dd8808eebe348'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.942267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f20bf-6a02-4dc7-9738-5580d69f9461 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:21.949674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d32dd096574f718c1dd8808eebe348'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.964413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:21.966184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:21.967232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:21.968860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:21.974388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:21.979966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:21.984250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d32dd096574f718c1dd8808eebe348'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:21.988282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d32dd096574f718c1dd8808eebe348'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:22.012137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d32dd096574f718c1dd8808eebe348'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:22.016592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:22.039435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 890/1788] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:21 2022] DELETE /identity/v3/users/59d32dd096574f718c1dd8808eebe348 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:22.081359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 899/1789] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:21 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:22.083328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:22.085350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:22.212262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfdf9f-1348-4b09-8444-af20f8cb9ff9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:22.240247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:22.379051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:22.381388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:22.382180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:22.382991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:22.384389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:22.389151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:22.432807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 891/1790] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:22 2022] POST /identity/v3/users => generated 291 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:22.438338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:22.470521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12b4c0-6436-4288-b4b9-061a32342b2d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:22.521936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:22.527666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:22.529853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:22.532550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:22.536775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:22.549511 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:22.605890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 900/1791] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:22.609863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:22.630210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f48-3e70-441d-a3ec-ee701ccb9027 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:22.645827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:22.646557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:22.646845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:22.647129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:22.647953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:22.649644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b422d7d-4491-412e-9adf-9cba0b0dba02 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:22.675124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 892/1792] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:22.678139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:22.709379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8dcc6-5071-44da-8524-ec415c86798b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:22.719787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:22.720953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:22.721590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:22.721769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] PATH_INFO: `/v3/projects/789001c6a08d4281b8e84a526803430a/users/3fc512c4788547709df89da7a21bb7c9/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:22.758462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] RBAC: Authorizing `identity:create_grant(project_id=789001c6a08d4281b8e84a526803430a, user_id=3fc512c4788547709df89da7a21bb7c9, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:22.762866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ee760-9f4e-4e21-b6d9-c022a1c5116c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:22.795115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 901/1793] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:22 2022] PUT /identity/v3/projects/789001c6a08d4281b8e84a526803430a/users/3fc512c4788547709df89da7a21bb7c9/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:22.807759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:22.808110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:22.808477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:22.861416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] MFA Rules not processed for user `3fc512c4788547709df89da7a21bb7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:22.865186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:23.035715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.036449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.037024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.048545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 893/1794] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:23.053563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.099761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] MFA Rules not processed for user `85bc6b75434c4e54afcf3645a46449eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:23.101300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:23.230852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 902/1795] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:23.239605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.249681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:23.266293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c61e1-9450-4811-bbb6-a096630a07cd None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.273758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01151bd0-ac62-459d-a34b-34b387548498 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.276569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.278731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.280138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.281039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.283372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.284266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.286460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.287438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.287832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.288208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.289901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.293894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.350522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 903/1796] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:23 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:23.365588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 894/1797] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:23 2022] POST /identity/v3/projects => generated 381 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:23.366534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.366841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.384612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:23.576051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e324e-e083-4eb4-bc18-9baef83b94cb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.611020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fd4-0cb3-4b91-823b-9412365b34a9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.621282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.622783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.623440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.624134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.625916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.630883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.670059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:23.683683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 895/1798] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:23 2022] POST /identity/v3/users => generated 297 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:23.687260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.722812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda90a-69b4-4999-869f-baa0f7a9535b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.735150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.737267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.737960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.738740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.741369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.745664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.768141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.772056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.773021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.773714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.775547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.779713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 896/1799] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:23.780738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.782080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.835353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97686d8-8c06-4ece-b033-d10b13611411 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.847898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 904/1800] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:23 2022] POST /identity/v3/users => generated 292 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:23.850009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.856215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:23.861242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.863065 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.863988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.864761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] PATH_INFO: `/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6/users/38b4a6268dc541e18ba416087c51e2bd/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.892196 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459ba5-c550-4aa5-8e37-b8cfd75f92ee admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:23.920677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1ebfba858a4aadb7e09c59b8807fe6, user_id=38b4a6268dc541e18ba416087c51e2bd, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.928157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:23.942096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:23.953928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:23.955751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:23.956639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] PATH_INFO: `/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:23.969689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 897/1801] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:23 2022] PUT /identity/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6/users/38b4a6268dc541e18ba416087c51e2bd/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:23.977107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:23.979932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93d612c2eb7404d811f0f883c4b1b5e)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:23.984210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.186396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93d612c2eb7404d811f0f883c4b1b5e', 'request_id': 'req-6448cd61-15b5-4719-9082-e96bac168fb8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:24.202136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93d612c2eb7404d811f0f883c4b1b5e', 'request_id': 'req-6448cd61-15b5-4719-9082-e96bac168fb8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:24.213873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fada05-884f-45d4-85c4-68bd76981281 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.214703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93d612c2eb7404d811f0f883c4b1b5e', 'request_id': 'req-6448cd61-15b5-4719-9082-e96bac168fb8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:24.250829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] The token cache is being invalidate because project e93d612c2eb7404d811f0f883c4b1b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:24.251324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:24.252752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:24.256665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 905/1802] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:23 2022] DELETE /identity/v3/projects/e93d612c2eb7404d811f0f883c4b1b5e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:24.258118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.275779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.360398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.361197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.361495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.361784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.362586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.365356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e17ded-8bab-46cc-ae5a-7abc044563ee admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.404921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 898/1803] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:24.408437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.409088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.409850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.416117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448cd61-15b5-4719-9082-e96bac168fb8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.423519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:24.424953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.426053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.430064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.431837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.432922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.434917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.450602 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 906/1804] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:24.451420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:24.452656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.454149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.455570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.475622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 899/1805] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:24.475842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab6206-105e-4154-a4b9-13ef0e66ed95 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.475981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.481767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.484021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.484399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.484758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] PATH_INFO: `/v3/projects/33fd54359bb44abb83c5885aa27b0509/users/9bbcf6e8bc574458bd2cf57eb78818ea/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.490268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16770658-e749-4375-aad6-a6e143f45b83 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.499928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.510907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.510907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.510907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] PATH_INFO: `/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6/users/38b4a6268dc541e18ba416087c51e2bd/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.511209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fd54359bb44abb83c5885aa27b0509, user_id=9bbcf6e8bc574458bd2cf57eb78818ea, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.511304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1ebfba858a4aadb7e09c59b8807fe6, user_id=38b4a6268dc541e18ba416087c51e2bd, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.511397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27662c32-55ae-49f4-b5a1-842011ab342e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.513252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.525867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 907/1806] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:24 2022] PUT /identity/v3/projects/33fd54359bb44abb83c5885aa27b0509/users/9bbcf6e8bc574458bd2cf57eb78818ea/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:24.527620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4879f-b300-454a-af8e-819877d8dede None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.528007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4879f-b300-454a-af8e-819877d8dede None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.528352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4879f-b300-454a-af8e-819877d8dede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.533718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 900/1807] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:24 2022] PUT /identity/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6/users/38b4a6268dc541e18ba416087c51e2bd/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:24.534814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.555206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-76c4879f-b300-454a-af8e-819877d8dede None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:24.556881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-76c4879f-b300-454a-af8e-819877d8dede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.568249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:24.625876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7299e32-5ac2-4376-b1f4-418e752d9e01 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.631395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.632216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.632510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.633090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.633254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 908/1808] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:24.633676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.635102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.635641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.635752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.636136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.651438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 901/1809] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:24.653201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.669004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a8c9-2572-41be-a2f8-2be331e8194f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.675354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.675861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] MFA Rules not processed for user `38b4a6268dc541e18ba416087c51e2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:24.676359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.676735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.677162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.677339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.678162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:24.678576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.680671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.699560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 902/1810] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:24.699560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.708391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916934dc-607a-43f2-90a8-9a5855cfb331 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.714246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.715548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.716084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.716340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] PATH_INFO: `/v3/projects/33fd54359bb44abb83c5885aa27b0509/users/9bbcf6e8bc574458bd2cf57eb78818ea/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.719072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fd54359bb44abb83c5885aa27b0509, user_id=9bbcf6e8bc574458bd2cf57eb78818ea, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.722090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.739092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 903/1811] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:24 2022] PUT /identity/v3/projects/33fd54359bb44abb83c5885aa27b0509/users/9bbcf6e8bc574458bd2cf57eb78818ea/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:24.740264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.799948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 909/1812] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:24.814811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.819180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:24.889338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03c83-0d47-4552-8cc6-b7abedcea491 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.896987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.898449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.898961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.899831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:24.900883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:24.914152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:24.918779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22012ec7-cc68-4df4-97e1-fe367671ab02 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:24.991770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 904/1813] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:24 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:24.993845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:24.994951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:24.996314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:24.996651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:24.996926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.019233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:25.095169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:25.155362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.157227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0974179-0280-43b3-9cc3-6b36ccf5b8b9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:25.157885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.171925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:25.173488 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.174191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.175304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.176745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.184376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 910/1814] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:25.184639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.246790 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 905/1815] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:24 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:25.248150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:25.264956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ae6ca-f1ad-4b05-96d6-88f08c031967 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:25.282800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:25.283577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.283843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.284183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.285453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.287183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.310093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 911/1816] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:25.315649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:25.335519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4e542-e269-467c-8093-e23d8228e530 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:25.353659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:25.354603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.356810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.357148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.358126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.363028 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.372356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:25.374596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:25.398747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 906/1817] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:25.404979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:25.430240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0989c7-0066-428c-bd7d-9d7007033255 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:25.444317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:25.445492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.445840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.446163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] PATH_INFO: `/v3/projects/8b82346d38d64c018de56c23581539bf/users/6f5278076b2c4a1c8d8763243df2908f/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.498400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9005f3-6020-4fca-8366-a68fae0b8b0c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:25.501304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b82346d38d64c018de56c23581539bf, user_id=6f5278076b2c4a1c8d8763243df2908f, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.503788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:25.505286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.505704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.505939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.506500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce34d3e-5916-496e-8bc5-f21ba8d763ec admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.509181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:25.564756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 907/1818] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:25 2022] PUT /identity/v3/projects/8b82346d38d64c018de56c23581539bf/users/6f5278076b2c4a1c8d8763243df2908f/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:25.585930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:25.585930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:25.586375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:25.637939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:25.644320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:25.684282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 912/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:25.686544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] MFA Rules not processed for user `6f5278076b2c4a1c8d8763243df2908f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:25.689195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:25.937124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 908/1820] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:25.944236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:25.971601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c811d-e313-4867-94ae-1c5ae149266d service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.113363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:26.139177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.141056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.142082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.142938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.145016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:26.149845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:26.204255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 913/1821] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:25 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:26.209257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:26.230046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107324ba-47bf-42c5-9d5c-ddc4683a332a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.377306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.380599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.385764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.386166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.388136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:26.392342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:26.452071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 909/1822] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:26 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:26.456757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:26.489779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7a1f9-f177-4e6a-9b3d-914b7aea6c86 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.501373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.502726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.503364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.503952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.505495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:26.513587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:26.536242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 914/1823] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:26.548201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:26.570548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb84a-9064-4f52-a241-5f9cdcb64402 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.577598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.578658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.579013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.579314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] PATH_INFO: `/v3/projects/432b5ed3481647a58bab5c5f09db001e/users/85243a3c8ee74947b969398a8b8cb945/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.604186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:26.628405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b5ed3481647a58bab5c5f09db001e, user_id=85243a3c8ee74947b969398a8b8cb945, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:26.636063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972cf61-81bd-4a0e-b4a1-028a8fab77fa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:26.689974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:26.690257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 910/1824] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:26 2022] PUT /identity/v3/projects/432b5ed3481647a58bab5c5f09db001e/users/85243a3c8ee74947b969398a8b8cb945/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:26.726673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6fca4-5a80-41bc-8e5a-c92dc3c5709d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.853243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.854182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.854952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.893393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.894068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.894728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.895024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.897273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:26.901060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:26.932615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 915/1825] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:26.934563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:26.953212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] MFA Rules not processed for user `9bbcf6e8bc574458bd2cf57eb78818ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:26.955217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:26.976038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd3f4c-3c81-462b-8f90-adab2860650b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:26.986006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:26.988506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:26.989257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:26.990784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] PATH_INFO: `/v3/projects/432b5ed3481647a58bab5c5f09db001e/users/85243a3c8ee74947b969398a8b8cb945/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:26.999404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b5ed3481647a58bab5c5f09db001e, user_id=85243a3c8ee74947b969398a8b8cb945, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:27.007291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25fbf6-9044-4831-977c-6e6b4efd04a5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:27.060651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 916/1826] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:26 2022] PUT /identity/v3/projects/432b5ed3481647a58bab5c5f09db001e/users/85243a3c8ee74947b969398a8b8cb945/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:27.077060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:27.078427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:27.079696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:27.142191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] MFA Rules not processed for user `85243a3c8ee74947b969398a8b8cb945`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:27.142408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 911/1827] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:27.144538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:27.172898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:27.275532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 917/1828] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:27.290456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e9245-a081-40d8-b1c5-17ce9d2e660e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:27.358064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:27.358761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:27.359042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:27.359338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:27.362475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:27.449547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:27.499890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:27.501813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:27.541545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 912/1829] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:27.632828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:27.647224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4a200-3164-4329-bfe5-9307577b5e04 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:27.658651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:27.659440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:27.659796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:27.660098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:27.679648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:27.681200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d411c-0ae0-468b-a57e-d2409ffebba1 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:27.704561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 918/1830] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:28.771621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:28.930116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f79efa-e876-4131-9bd8-6f1130784842 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:28.936057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:28.938604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:28.939235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:28.939390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] PATH_INFO: `/v3/users/6f5278076b2c4a1c8d8763243df2908f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:28.941114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5278076b2c4a1c8d8763243df2908f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:28.943653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:29.052217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5278076b2c4a1c8d8763243df2908f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:29.075109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5278076b2c4a1c8d8763243df2908f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:29.112408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5278076b2c4a1c8d8763243df2908f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:29.114037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5278076b2c4a1c8d8763243df2908f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:29.137403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5278076b2c4a1c8d8763243df2908f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:29.159895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 913/1831] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:28 2022] DELETE /identity/v3/users/6f5278076b2c4a1c8d8763243df2908f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:29.162062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:29.162835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:29.162835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:29.188461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:29.190445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:29.207369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:29.295330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 919/1832] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:29.316746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:29.319098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:29.370240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:29.372396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:29.419659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b1af8-0687-4cb3-8c23-19bcc3768cca admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:29.424179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:29.424877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:29.425154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:29.425427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:29.436157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:29.437994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:29.450347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 914/1833] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:29.476937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa724b-cf17-440b-b961-0576dbe3830d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:29.483546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:29.484622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:29.485112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:29.485578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:29.488675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:29.606112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:29.607651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:29.631675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 920/1834] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:30.133796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:30.136919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:30.154648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:30.163233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f4d75-12a8-41c2-b009-1e458bc51888 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:30.180859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:30.182541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:30.183094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:30.184025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] PATH_INFO: `/v3/projects/8b82346d38d64c018de56c23581539bf` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:30.186269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b82346d38d64c018de56c23581539bf)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:30.189385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:30.254561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b82346d38d64c018de56c23581539bf', 'request_id': 'req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.265538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b82346d38d64c018de56c23581539bf', 'request_id': 'req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.274224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b82346d38d64c018de56c23581539bf', 'request_id': 'req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.304922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] The token cache is being invalidate because project 8b82346d38d64c018de56c23581539bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:30.305551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.307083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.309589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 915/1835] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:30 2022] DELETE /identity/v3/projects/8b82346d38d64c018de56c23581539bf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:30.315564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:30.334596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8ca8f-9989-4f36-b162-b9cf33226b29 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:30.444775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:30.446254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:30.446710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:30.447344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] PATH_INFO: `/v3/users/85243a3c8ee74947b969398a8b8cb945` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:30.449576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] RBAC: Authorizing `identity:delete_user(user_id=85243a3c8ee74947b969398a8b8cb945)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:30.453663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:30.466682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:30.511641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85243a3c8ee74947b969398a8b8cb945'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.520739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85243a3c8ee74947b969398a8b8cb945'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.521869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85243a3c8ee74947b969398a8b8cb945'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.537405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85243a3c8ee74947b969398a8b8cb945'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.546679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-54f34930-a816-4249-b12d-b02c933adadf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85243a3c8ee74947b969398a8b8cb945'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:30.554423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 921/1836] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:30 2022] DELETE /identity/v3/users/85243a3c8ee74947b969398a8b8cb945 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:30.989735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:30.992261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:31.016950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc63e0-bd6f-4d38-a2f7-2805179ee822 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.120803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.121772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.122057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.122370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] PATH_INFO: `/v3/projects/432b5ed3481647a58bab5c5f09db001e` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.123793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432b5ed3481647a58bab5c5f09db001e)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.125797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.197597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432b5ed3481647a58bab5c5f09db001e', 'request_id': 'req-23c69156-61c4-4194-b709-33383ddcef00'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:31.206672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432b5ed3481647a58bab5c5f09db001e', 'request_id': 'req-23c69156-61c4-4194-b709-33383ddcef00'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:31.217059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432b5ed3481647a58bab5c5f09db001e', 'request_id': 'req-23c69156-61c4-4194-b709-33383ddcef00'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:31.229371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] The token cache is being invalidate because project 432b5ed3481647a58bab5c5f09db001e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:31.229687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:31.230385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-23c69156-61c4-4194-b709-33383ddcef00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:31.232441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 916/1837] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:30 2022] DELETE /identity/v3/projects/432b5ed3481647a58bab5c5f09db001e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:31.238402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906a355-fc71-46f7-9232-63a675846589 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.238888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906a355-fc71-46f7-9232-63a675846589 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.239359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906a355-fc71-46f7-9232-63a675846589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.267340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a906a355-fc71-46f7-9232-63a675846589 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:31.269441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a906a355-fc71-46f7-9232-63a675846589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:31.377953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 922/1838] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:31.390040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.390347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.390730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.419095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:31.420522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:31.435546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 917/1839] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:31.439670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906a355-fc71-46f7-9232-63a675846589 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.452545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906a355-fc71-46f7-9232-63a675846589 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.458536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.459560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.461813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.462237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.463722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:31.464177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.466203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.482875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 923/1840] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:16:31.489250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.517167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0fd19-01fb-48c4-8886-5102ff9ab23f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.525266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.525981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.526266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.526547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.527342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.530527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.544537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:31.574993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 918/1841] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:31 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:31.581282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.609547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1727d-c742-40a0-a597-37118c7e0ec3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.683571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.684397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.684771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.685114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.685907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.688061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.714107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 924/1842] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:31 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:31.721142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.740333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd16b8-5714-4bdd-8213-031c3adcebab admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.748544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.749713 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.750183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.750660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.752185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.755297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.774586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 919/1843] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:31.779305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.786367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:31.791273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c571b-fd39-49d8-aea9-a977556a67c5 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.798043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.799585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.800107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.801459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.802848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.806024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c8d2b-c662-4a59-9563-625664e07bf1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.818475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 925/1844] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:31.823176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:31.835179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b839b0-2eeb-4612-8930-e99fc680ba3d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:31.841414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:31.842397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.842990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.843317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] PATH_INFO: `/v3/projects/1c7cc165ff714c57a267461d67acef84/users/cc71f45ba14345b0bcc593f2629e521e/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.862926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7cc165ff714c57a267461d67acef84, user_id=cc71f45ba14345b0bcc593f2629e521e, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:31.866231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:31.882859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 920/1845] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:31 2022] PUT /identity/v3/projects/1c7cc165ff714c57a267461d67acef84/users/cc71f45ba14345b0bcc593f2629e521e/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:31.901104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:31.901416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:31.901714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:31.935412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] MFA Rules not processed for user `cc71f45ba14345b0bcc593f2629e521e`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:31.936712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:32.015083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 926/1846] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:32.027595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:32.029559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:32.120748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbe52f-34de-462f-96a3-a9ad2bc8f6ec admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:32.127080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:32.128219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:32.128706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:32.129202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:32.146260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:32.148521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76aee4-eeb7-4026-941e-bcc518e64d95 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:32.165113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 921/1847] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:32.185305 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:32.283160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97ecaf-fbc5-47ca-bbde-193114e4fc91 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:32.290653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:32.291751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:32.292124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:32.292489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] PATH_INFO: `/v3/users/cc71f45ba14345b0bcc593f2629e521e` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:32.294061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc71f45ba14345b0bcc593f2629e521e)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:32.296129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:32.363030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc71f45ba14345b0bcc593f2629e521e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:32.372438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc71f45ba14345b0bcc593f2629e521e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:32.373320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc71f45ba14345b0bcc593f2629e521e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:32.393886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc71f45ba14345b0bcc593f2629e521e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:32.406298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc71f45ba14345b0bcc593f2629e521e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:32.417765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 927/1848] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:32 2022] DELETE /identity/v3/users/cc71f45ba14345b0bcc593f2629e521e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:32.425838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:32.426382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:32.427038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:32.449019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:32.450774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:32.603322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 922/1849] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:32.621698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:32.624601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:32.635786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:32.865255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bc3a4-418c-4f61-9da8-3b2ce746d87e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:32.873135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:32.875829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:32.876450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:32.877058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:32.899551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:32.902957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d1b2-b6c0-41ed-8fb5-2e32c4e446e9 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:32.928201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 928/1850] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:33.811717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:33.814237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:33.838552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3188721-0c77-411c-add6-7e8a17ffd4ed None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:33.855766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:33.857320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:33.857747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:33.858124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] PATH_INFO: `/v3/projects/1c7cc165ff714c57a267461d67acef84` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:33.860584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7cc165ff714c57a267461d67acef84)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:33.863041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:33.973862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7cc165ff714c57a267461d67acef84', 'request_id': 'req-4ac4f24f-6128-4953-bc6b-93437b9fad12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:33.978086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:34.001723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7cc165ff714c57a267461d67acef84', 'request_id': 'req-4ac4f24f-6128-4953-bc6b-93437b9fad12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:34.026171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7cc165ff714c57a267461d67acef84', 'request_id': 'req-4ac4f24f-6128-4953-bc6b-93437b9fad12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:34.091397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] The token cache is being invalidate because project 1c7cc165ff714c57a267461d67acef84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:34.091872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:34.093213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4ac4f24f-6128-4953-bc6b-93437b9fad12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:34.097825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 923/1851] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:33 2022] DELETE /identity/v3/projects/1c7cc165ff714c57a267461d67acef84 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:34.104427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:34.105740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:34.106135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:34.143515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:34.145038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:34.288881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 929/1852] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:34.303969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:34.306434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:34.307024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:34.368495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:34.371890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:34.392816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 924/1853] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:34.394521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:34.421022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feea383-c90d-4fe6-a569-32b6335ffa67 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:34.432745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:34.435310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:34.435653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:34.437116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:34.437942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:34.438260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:34.442882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:34.492008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 930/1854] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:34.498821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:34.520293 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0554ef2-6604-44aa-b324-85064c275b2d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:34.529462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:34.531922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:34.532691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:34.533403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:34.534933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:34.538817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:34.648804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 925/1855] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:34 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:34.653535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:34.676561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093ca35-0046-48d1-bd51-4e55e178f8ef admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:34.940008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:34.943793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:34.944250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:34.944419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:34.947223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:34.952595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:35.018431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 931/1856] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:34 2022] POST /identity/v3/users => generated 287 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:35.020229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:35.042609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724906c8-b670-40c2-b96b-965312465d37 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:35.069457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:35.072475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.082812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.082812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.082812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:35.088451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:35.177825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 926/1857] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:35.180396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:35.194879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63f2a3-98e4-4aa8-ab87-8967b2846082 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:35.228194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:35.230925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.233281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.233610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.239448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:35.245647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9966c6-2396-44cf-a1fa-ea892f7661a7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:35.291870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 932/1858] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:35.296830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:35.359363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76adc9-6f4b-4b6e-934f-c7c91260d896 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:35.374876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:35.377960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.378328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.378613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] PATH_INFO: `/v3/projects/f63544286a174b3d85dc0654b008148c/users/9d2dcdffd5dd47bc87fed46edada9fe3/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.421094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63544286a174b3d85dc0654b008148c, user_id=9d2dcdffd5dd47bc87fed46edada9fe3, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:35.423989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b4fa5-bad2-48b1-91df-27d4c20ef48e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:35.454174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 927/1859] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:35 2022] PUT /identity/v3/projects/f63544286a174b3d85dc0654b008148c/users/9d2dcdffd5dd47bc87fed46edada9fe3/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:35.476662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.476825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.477123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.552281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:35.580203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] MFA Rules not processed for user `9d2dcdffd5dd47bc87fed46edada9fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:35.583706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f8d25-40a3-43cb-a826-b124e40db434 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.584551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f8d25-40a3-43cb-a826-b124e40db434 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.584772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f8d25-40a3-43cb-a826-b124e40db434 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.586673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 928/1860] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:16:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:16:35.592380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:35.620681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:35.621005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:35.621281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:35.680420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:35.681878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:35.811572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 933/1861] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:35.930103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:35.934216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:36.009465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 929/1862] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:16:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:36.110813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5a0f9-b51b-498a-a55c-1d5acb728f04 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:36.128315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:36.129414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:36.129869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:36.130296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:36.162568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:36.172906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:36.181644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:36.186939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:36.208403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6b2bb-8187-4020-b942-063a336d5697 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:36.216472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 934/1863] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:36.228413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:36.230922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:36.232586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:36.233806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:36.266545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:36.275446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba6bdd-0610-450a-b24b-361e8ddb3923 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:36.344531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 930/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:38.277386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:38.279155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:38.433843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c0571-9110-41c7-9f91-3adb2fd48b59 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:38.440731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:38.442389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:38.442874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:38.443095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] PATH_INFO: `/v3/users/3fc512c4788547709df89da7a21bb7c9` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:38.444901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc512c4788547709df89da7a21bb7c9)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:38.447669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:38.525925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:38.527831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc512c4788547709df89da7a21bb7c9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:38.538649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc512c4788547709df89da7a21bb7c9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:38.540007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc512c4788547709df89da7a21bb7c9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:38.564309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc512c4788547709df89da7a21bb7c9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:38.580283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc512c4788547709df89da7a21bb7c9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:38.591401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 935/1865] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:38 2022] DELETE /identity/v3/users/3fc512c4788547709df89da7a21bb7c9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:38.598103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:38.598682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:38.599111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:38.645037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:38.647176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:38.751078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:38.784266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 931/1866] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:38.804876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:38.808228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:38.904432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84f05-c91a-4fd8-a881-217d8b462c08 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:38.909920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:38.911044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:38.911670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:38.912056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:38.925133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:38.927058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:38.943766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 936/1867] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:39.917004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:39.920717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:39.962417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f4b76-923e-4f7a-9db7-6e8f29c0a9c7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:39.976097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:39.978471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:39.979247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:39.980280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] PATH_INFO: `/v3/projects/789001c6a08d4281b8e84a526803430a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:39.982952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] RBAC: Authorizing `identity:delete_project(project_id=789001c6a08d4281b8e84a526803430a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:39.988166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:40.088903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789001c6a08d4281b8e84a526803430a', 'request_id': 'req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.100177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789001c6a08d4281b8e84a526803430a', 'request_id': 'req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.108117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789001c6a08d4281b8e84a526803430a', 'request_id': 'req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.125744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] The token cache is being invalidate because project 789001c6a08d4281b8e84a526803430a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:40.126034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.126754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.128450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 932/1868] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:39 2022] DELETE /identity/v3/projects/789001c6a08d4281b8e84a526803430a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:40.132810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:40.143368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383a98-ec6d-4f67-ac57-745c47b866a3 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:40.206727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:40.207659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:40.207939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:40.208210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] PATH_INFO: `/v3/users/38b4a6268dc541e18ba416087c51e2bd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:40.209564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b4a6268dc541e18ba416087c51e2bd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:40.211741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:40.266388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b4a6268dc541e18ba416087c51e2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.275380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b4a6268dc541e18ba416087c51e2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.276226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b4a6268dc541e18ba416087c51e2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.284644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:40.290911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b4a6268dc541e18ba416087c51e2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.302266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b654965d-bc9c-4f18-9713-bb992b1bc112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b4a6268dc541e18ba416087c51e2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.310144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 937/1869] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:40 2022] DELETE /identity/v3/users/38b4a6268dc541e18ba416087c51e2bd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:40.695726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:40.699603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:40.741173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b791-5d78-4dc6-83dd-fcc500f3563a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:40.828902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:40.829732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:40.829982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:40.830259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] PATH_INFO: `/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:40.831635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1ebfba858a4aadb7e09c59b8807fe6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:40.833616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:40.895093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1ebfba858a4aadb7e09c59b8807fe6', 'request_id': 'req-a88b3834-65e8-486f-9683-f5a470afa90a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.906023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1ebfba858a4aadb7e09c59b8807fe6', 'request_id': 'req-a88b3834-65e8-486f-9683-f5a470afa90a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.921215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1ebfba858a4aadb7e09c59b8807fe6', 'request_id': 'req-a88b3834-65e8-486f-9683-f5a470afa90a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.935327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] The token cache is being invalidate because project 4c1ebfba858a4aadb7e09c59b8807fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:40.935656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.936339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a88b3834-65e8-486f-9683-f5a470afa90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:40.940944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 933/1870] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:40 2022] DELETE /identity/v3/projects/4c1ebfba858a4aadb7e09c59b8807fe6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:40.947687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:40.948515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:40.949483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:40.990181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:40.992431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:41.108236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 938/1871] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:41.119453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.119944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.120574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.185113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:41.186543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:41.200705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 934/1872] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:41.205362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.228701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edca72-c309-4b81-9f81-b7ebe978a675 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.234759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.235513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.235879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.236164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.236942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:41.237258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.238940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.254335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 939/1873] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:41.259342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.264759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:41.273050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364d19-2524-48e5-82f8-1e6fb9ba85e9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.280540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.281722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.282083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.282477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.283266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.286261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.321681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 935/1874] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:41.326957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.338759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a8163-75e8-4b0e-9a0a-62f557e3c6ca admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.421693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.422487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.422930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.423281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.424228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.427267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.456582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 940/1875] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:41.460087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.476156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08d6d8-2f12-4a67-97b5-5be34c061e24 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.484721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.485888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.486374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.486859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.488288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.491441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.517903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 936/1876] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:41.519885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.538108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7282822-0209-484a-b5f5-9cfc7c6d1208 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.546459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.547642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.548169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.548664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.550139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.553410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c3505-d2a7-459f-b980-f7b0224f70ab admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.573358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 941/1877] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:41.578127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.589845 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda014-3cc4-4e8c-bfbc-7ccbbdbd3941 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.595346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.596741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.597116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.597590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] PATH_INFO: `/v3/projects/abd74abab12d445bb85867106972cf21/users/a5483361741b4ae5b50bf6ece7fe6dbc/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.620138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd74abab12d445bb85867106972cf21, user_id=a5483361741b4ae5b50bf6ece7fe6dbc, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.626200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.648197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 937/1878] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:41 2022] PUT /identity/v3/projects/abd74abab12d445bb85867106972cf21/users/a5483361741b4ae5b50bf6ece7fe6dbc/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:41.659663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.659975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.660356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.691546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] MFA Rules not processed for user `a5483361741b4ae5b50bf6ece7fe6dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:41.693166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:41.756259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:41.810836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 942/1879] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:41.812897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:41.932545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779486fe-0976-4e88-adaf-36962f93784c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:41.938359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:41.939345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:41.939872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:41.940286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:41.941318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:41.944764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:41.987982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 938/1880] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:41.989197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.004309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183c8c8-de1b-421a-83e5-d3f676c59da3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.093786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.094677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.094962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.095297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.096421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.099970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.124787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 943/1881] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:41 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:16:42.128787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.139867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6c50a-3ecb-473f-9bd0-c45f033432fa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.145514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.146289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.146663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.146953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.148115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.150125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.171794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 939/1882] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:42.173452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.186852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62d68-e0ec-488a-a837-b0f177177df4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.193696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.194831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.195262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.195582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] PATH_INFO: `/v3/projects/39c4269b3628444bb2a98eefc41e5e8b/users/966bdffe9aeb423384f4edf13c32a473/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.219379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c4269b3628444bb2a98eefc41e5e8b, user_id=966bdffe9aeb423384f4edf13c32a473, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.223131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.259176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 944/1883] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:42 2022] PUT /identity/v3/projects/39c4269b3628444bb2a98eefc41e5e8b/users/966bdffe9aeb423384f4edf13c32a473/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:42.265996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.282112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f698e-0fe4-4b75-845d-b9aef3dd1789 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.361447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.362159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.362445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.362726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.363521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.365208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155391b-e0cb-4a4e-9eaf-480b182f2580 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.377215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 940/1884] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:42.383616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.403345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997ac87-578a-4241-9f67-b6ef418305b4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.411553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.412836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.413171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.413487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] PATH_INFO: `/v3/projects/39c4269b3628444bb2a98eefc41e5e8b/users/966bdffe9aeb423384f4edf13c32a473/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.416992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39c4269b3628444bb2a98eefc41e5e8b, user_id=966bdffe9aeb423384f4edf13c32a473, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.420149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.435253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 945/1885] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:42 2022] PUT /identity/v3/projects/39c4269b3628444bb2a98eefc41e5e8b/users/966bdffe9aeb423384f4edf13c32a473/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:42.447158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.447445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.447764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.482692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] MFA Rules not processed for user `966bdffe9aeb423384f4edf13c32a473`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:42.484009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:42.488693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:42.564263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 941/1886] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:42.581463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:42.583398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:42.655714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90367fb-e746-4d18-9a14-40f25b03b644 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:42.661127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:42.661644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:42.661934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:42.662203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:42.673564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:42.675076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:42.689554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 946/1887] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:42 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:43.372221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:43.373965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:43.582746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:43.599397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf551-aa3f-4c88-be39-32e29e44d4f5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:43.602319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca46d-05cd-4696-a102-ca5cf273d6ce service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:43.615562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:43.617491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:43.617726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:43.617850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:43.618882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:43.619329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:43.619758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] PATH_INFO: `/v3/users/9d2dcdffd5dd47bc87fed46edada9fe3` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:43.620039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] PATH_INFO: `/v3/users/a5483361741b4ae5b50bf6ece7fe6dbc` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:43.621642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5483361741b4ae5b50bf6ece7fe6dbc)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:43.623226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2dcdffd5dd47bc87fed46edada9fe3)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:43.624011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:43.628758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:43.712338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:43.739937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5483361741b4ae5b50bf6ece7fe6dbc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.753948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5483361741b4ae5b50bf6ece7fe6dbc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.755380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5483361741b4ae5b50bf6ece7fe6dbc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.773164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2dcdffd5dd47bc87fed46edada9fe3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.791691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5483361741b4ae5b50bf6ece7fe6dbc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.794661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2dcdffd5dd47bc87fed46edada9fe3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.815438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-67ca4d1a-70af-4d58-81ec-26754cd58570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5483361741b4ae5b50bf6ece7fe6dbc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.820114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2dcdffd5dd47bc87fed46edada9fe3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.820900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2dcdffd5dd47bc87fed46edada9fe3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.831513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 947/1888] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:43 2022] DELETE /identity/v3/users/a5483361741b4ae5b50bf6ece7fe6dbc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:43.831716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7eaa4a0-7d5e-4b09-89fc-a836651dc1df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2dcdffd5dd47bc87fed46edada9fe3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:43.835836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:43.836228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:43.836518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:43.842775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 942/1889] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:43 2022] DELETE /identity/v3/users/9d2dcdffd5dd47bc87fed46edada9fe3 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:43.845565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:43.845870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:43.846130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:43.869753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:43.871425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:43.872798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:43.873678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:43.891034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:43.999412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 948/1890] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:44.000067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 943/1891] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:44.011173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:44.015883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:44.021853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:44.025116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:44.119368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d8326-1498-4609-aabe-877712cf9a7c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:44.124312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:44.124988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:44.125267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:44.125563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:44.141877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f525-1f26-485a-be31-b614b80260f3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:44.163783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:44.165953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:44.166089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:44.166709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:44.167000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:44.167292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:44.178428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:44.179636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 949/1892] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:44.180364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:44.194363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 944/1893] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:45.119530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.122282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.155377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.158470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.202474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30141c7-3ad5-4a7c-a97a-3634cd1a1669 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:45.205575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040af5e-0cd4-4e64-a9e9-11c9c7d95285 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:45.212864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:45.214543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.215043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.215471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] PATH_INFO: `/v3/projects/f63544286a174b3d85dc0654b008148c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.218287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63544286a174b3d85dc0654b008148c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:45.221728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:45.233986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:45.235629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.236150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.236732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] PATH_INFO: `/v3/projects/abd74abab12d445bb85867106972cf21` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.247373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd74abab12d445bb85867106972cf21)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:45.251300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:45.337888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:45.338569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63544286a174b3d85dc0654b008148c', 'request_id': 'req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.342074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd74abab12d445bb85867106972cf21', 'request_id': 'req-d720e063-7e66-4ba2-a47e-8c8647f776f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.350553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd74abab12d445bb85867106972cf21', 'request_id': 'req-d720e063-7e66-4ba2-a47e-8c8647f776f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.354524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63544286a174b3d85dc0654b008148c', 'request_id': 'req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.358437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd74abab12d445bb85867106972cf21', 'request_id': 'req-d720e063-7e66-4ba2-a47e-8c8647f776f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.374824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63544286a174b3d85dc0654b008148c', 'request_id': 'req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.396638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] The token cache is being invalidate because project abd74abab12d445bb85867106972cf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:45.397586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.398904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.400890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] The token cache is being invalidate because project f63544286a174b3d85dc0654b008148c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:45.401403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.402274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f6a8313e-3d0a-4515-9ed2-ea0442bf75d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.403643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 950/1894] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:45 2022] DELETE /identity/v3/projects/abd74abab12d445bb85867106972cf21 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:45.404612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 945/1895] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:45 2022] DELETE /identity/v3/projects/f63544286a174b3d85dc0654b008148c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:45.408979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.412696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004750e-bb2a-4a38-9b0b-5e431b0e110c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.412696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004750e-bb2a-4a38-9b0b-5e431b0e110c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.413637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.413959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004750e-bb2a-4a38-9b0b-5e431b0e110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.443383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1004750e-bb2a-4a38-9b0b-5e431b0e110c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:45.444885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1004750e-bb2a-4a38-9b0b-5e431b0e110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.503103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:45.529775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 946/1896] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:45.538295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.538581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.538857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.551104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e063-7e66-4ba2-a47e-8c8647f776f3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:45.557933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:45.558764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:45.559065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.559684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.559850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] PATH_INFO: `/v3/users/966bdffe9aeb423384f4edf13c32a473` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.560111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.561328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966bdffe9aeb423384f4edf13c32a473)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:45.563509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:45.570598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 947/1897] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:45.574858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.587175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f171f0-d1b6-436b-81ac-5d3a718a35b2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:45.592399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:45.593108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.593396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.593674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.594434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:45.594742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:45.596518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:45.610068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 948/1898] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:16:45.615978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.631101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966bdffe9aeb423384f4edf13c32a473'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.640220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e969ed9-6abe-497f-8a13-b439d5afd04f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:45.643063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966bdffe9aeb423384f4edf13c32a473'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.643877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966bdffe9aeb423384f4edf13c32a473'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.650929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:45.654353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:45.654919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:45.656189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:45.657785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:45.662572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:45.662886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966bdffe9aeb423384f4edf13c32a473'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.674498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966bdffe9aeb423384f4edf13c32a473'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:45.697665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 951/1899] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:45 2022] DELETE /identity/v3/users/966bdffe9aeb423384f4edf13c32a473 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:45.728942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 949/1900] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:45 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:45.730321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:45.731963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:45.884001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:45.903589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164187a6-6bf7-44ab-a489-e2b6de46a5b1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:46.170061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:46.171330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.174803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.175621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.177033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:46.185641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:46.257672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 952/1901] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:45 2022] POST /identity/v3/users => generated 289 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:46.271544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:46.308508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364908be-6429-4a35-af97-663b64450ac9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:46.334669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:46.336228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.336907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.342208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.344720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:46.354139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:46.400692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 950/1902] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:46.412252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:46.452158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08afd9-b0ce-4f8a-9805-28adcd249d4c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:46.484982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:46.486478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.487155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.487886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.489847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:46.494013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:46.551417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 953/1903] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:46.559646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:46.613963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3b18c-8781-4a01-8358-91cee5d6b17c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:46.630121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:46.631927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.632422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.632916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] PATH_INFO: `/v3/projects/1decfa476faf415581dacad34b31320c/users/19c2c9e691d2482a906c230096e56bae/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.663651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1decfa476faf415581dacad34b31320c, user_id=19c2c9e691d2482a906c230096e56bae, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:46.666604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7c4b6-3feb-497d-8da7-5039e738a573 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:46.692982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 951/1904] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:46 2022] PUT /identity/v3/projects/1decfa476faf415581dacad34b31320c/users/19c2c9e691d2482a906c230096e56bae/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:46.698564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:46.701529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:46.705575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.706044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.706620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.762756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] MFA Rules not processed for user `19c2c9e691d2482a906c230096e56bae`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:46.763727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93f286-97b0-4553-b579-ccbf56c3a81f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:46.764761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:46.942382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 952/1905] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:46.944004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:46.952161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:46.953803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:46.954160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:46.954506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] PATH_INFO: `/v3/projects/39c4269b3628444bb2a98eefc41e5e8b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:46.956549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=39c4269b3628444bb2a98eefc41e5e8b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:46.958575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.039655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39c4269b3628444bb2a98eefc41e5e8b', 'request_id': 'req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:47.052051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39c4269b3628444bb2a98eefc41e5e8b', 'request_id': 'req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:47.060385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39c4269b3628444bb2a98eefc41e5e8b', 'request_id': 'req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:47.070966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606bb2-9e98-4a09-8f0f-dcb2cd7b14ee None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.075128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:47.075410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] The token cache is being invalidate because project 39c4269b3628444bb2a98eefc41e5e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:47.075860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:47.076615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-777d48ab-7fc2-4f22-8561-00b1dffdc3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:47.078439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 954/1906] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:46 2022] DELETE /identity/v3/projects/39c4269b3628444bb2a98eefc41e5e8b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:47.083634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f6a7-7ee3-465e-96b6-971a2c354fa4 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.083925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f6a7-7ee3-465e-96b6-971a2c354fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.084211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5f6a7-7ee3-465e-96b6-971a2c354fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.105779 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-10f5f6a7-7ee3-465e-96b6-971a2c354fa4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:47.107040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-10f5f6a7-7ee3-465e-96b6-971a2c354fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:47.112733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:47.196971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.198253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.205709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 955/1907] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:47.206543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.206543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.206543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.206970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.208969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.210735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.211221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.248935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:47.250969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:47.252375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 953/1908] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:46 2022] POST /identity/v3/projects => generated 375 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:47.260449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.262631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:47.359676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 956/1909] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:47.362575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.374192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370589d-6b03-40de-a8e4-0bfef40a0fa3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.381222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.381571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f848e-a304-4968-87ae-2e8dbdeef59d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.382314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.382909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.383264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.384431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:47.384896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.387074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.387669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.388477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.388825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.389170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.389978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.392182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.412270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 957/1910] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:47.414463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.423774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 954/1911] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:47.429613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.437500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959e437-c593-436d-b024-9f07faab9d97 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.445950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.447119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.447665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.448143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.449264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55864373-113d-4a9a-8516-ee449b03e03b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.449474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.453207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.455162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.455985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.456416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.456719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.457628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.459401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.472511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 955/1912] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:47.476631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.493939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b328c-af5d-4a1b-a37d-867854eb2259 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.507240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 958/1913] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:47.508252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.509608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.521645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.524073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.524824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.527583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.529561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.563199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 956/1914] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:47.572225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.595909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5ea07-2104-4596-bc9e-34813b4f4d35 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.654519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5c51c-3b57-4347-9aea-b81712882047 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.672585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.673306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.673594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.673878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.674616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.676671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.723544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 959/1915] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:47.726826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.733735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:47.739080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b3eb-975e-4f83-a30e-fa3d5442936a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.747350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.748747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.749486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.750672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.751804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.754242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.758890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.759988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.760640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.761100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] PATH_INFO: `/v3/projects/1d56f71e51c64a07badb794c54ffaf97/users/eacae3c80f7744a7b3da09f0f991ef94/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.765391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:47.776180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 960/1916] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:47.780497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.796799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b042551-e6c2-42bd-9e55-a035abf0300f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.801943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d56f71e51c64a07badb794c54ffaf97, user_id=eacae3c80f7744a7b3da09f0f991ef94, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.809062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:47.809942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.810354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.810733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.811512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56933d-9070-4619-9722-5fcb1bf97b9e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.811793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:47.814308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:47.837705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 961/1917] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:47.844002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:47.852156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 957/1918] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:47 2022] PUT /identity/v3/projects/1d56f71e51c64a07badb794c54ffaf97/users/eacae3c80f7744a7b3da09f0f991ef94/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:47.873592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:47.873592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:47.873894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:47.879785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d41288-6c3e-4524-b47c-ba444fe8feca admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:47.966632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] MFA Rules not processed for user `eacae3c80f7744a7b3da09f0f991ef94`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:47.967998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:48.045388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.046833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.047175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.047465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] PATH_INFO: `/v3/projects/3155602ef2794a7cb86556e595460b22/users/ed94855f6cf64bd69435b32d375847ad/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.076551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3155602ef2794a7cb86556e595460b22, user_id=ed94855f6cf64bd69435b32d375847ad, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.106652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df87a734-5251-42ad-81c8-1c48fba52ee1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.169452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 958/1919] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:48.174403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 962/1920] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:47 2022] PUT /identity/v3/projects/3155602ef2794a7cb86556e595460b22/users/ed94855f6cf64bd69435b32d375847ad/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:48.176902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.199563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.199563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.200004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.240332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b335-ac64-4d2f-b089-b8fe60fde04b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.314666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] MFA Rules not processed for user `ed94855f6cf64bd69435b32d375847ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:48.317411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:48.449598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.450358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.451594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.451889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.452640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.454788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.521303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 959/1921] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:48 2022] POST /identity/v3/projects => generated 377 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:48.525421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.541763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 963/1922] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:48.548904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.569303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef36a-b357-44cc-adc5-b4c096570f53 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.628440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:48.635788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6143bf-412e-4efa-9338-ab8172bf1190 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.643967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.645002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.645613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.646090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.653775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.658584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.661413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:48.677916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.679042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.679334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.679635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.680367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.682570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.700347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 960/1923] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:48 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:48.702328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.706398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:48.721229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23a4d1-06a2-4208-8dd4-d80fb9dc7563 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.726610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.727608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 964/1924] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:48 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:48.728521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.731140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.731513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.731787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] PATH_INFO: `/v3/users/2bcb9805113f4bd280fd1a25f5e0b29d` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.733222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bcb9805113f4bd280fd1a25f5e0b29d)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.735236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.799351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bcb9805113f4bd280fd1a25f5e0b29d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:48.809857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bcb9805113f4bd280fd1a25f5e0b29d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:48.814186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4475-855c-4b1e-ae43-a783b089ffea admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.823549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.824592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.824876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.825174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:48.826312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:48.830526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:48.835957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bcb9805113f4bd280fd1a25f5e0b29d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:48.837358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bcb9805113f4bd280fd1a25f5e0b29d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:48.849198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bcb9805113f4bd280fd1a25f5e0b29d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:48.851519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 965/1925] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:48.853183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.856667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:48.863736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 961/1926] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:48 2022] DELETE /identity/v3/users/2bcb9805113f4bd280fd1a25f5e0b29d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:48.864901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:48.869420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:48.885824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2b44e-9c89-4f83-ba80-334c251f6400 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.984011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d988d-dc39-44fc-b892-35ead3626521 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:48.993887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:48.996500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:48.996980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:48.997270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] PATH_INFO: `/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7/users/13a978dd346f45f8b2a92c9743f36f8f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.039685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.041542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.042085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.042784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.044313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.045426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf03ca75d32149be8b6d6577f4bb08c7, user_id=13a978dd346f45f8b2a92c9743f36f8f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.048275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.052181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c96e-383a-4b07-b082-ab3d64733dd2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.072572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 962/1927] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:48 2022] PUT /identity/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7/users/13a978dd346f45f8b2a92c9743f36f8f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:49.074532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.074839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.075213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.086792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 966/1928] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:48 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:49.089403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.098684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:49.100244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:49.113013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dff31-05ec-4ddc-aeff-ccf3288a4c3a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.162044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:49.203618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 963/1929] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:49.204802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.210788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.212093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.212680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.213258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.215381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.219984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.223332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e9f14-6c97-4cb4-aca4-da5941213461 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.231153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.232352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.232919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.233317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.234729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.238238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.238725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 967/1930] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:49.245176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.247875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:49.263517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:49.265382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 964/1931] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:49.267668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.293035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fad8c5-0e84-4d20-af9a-aa55dd608e4c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.302200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.303911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.304271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.304621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] PATH_INFO: `/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7/users/13a978dd346f45f8b2a92c9743f36f8f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.307136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf03ca75d32149be8b6d6577f4bb08c7, user_id=13a978dd346f45f8b2a92c9743f36f8f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.310926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.328161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 965/1932] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:49 2022] PUT /identity/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7/users/13a978dd346f45f8b2a92c9743f36f8f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:49.329333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.460884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc65ca-5b51-4cdd-a9f7-207471632962 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.522234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52e763-f32c-4132-aebe-8eb69a893e39 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.536291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.537409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.537783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.538158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] PATH_INFO: `/v3/projects/5db4757dafc747069d0bb42fa2858601/users/bac58f88348a4a0fb1196afc354df351/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.581431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db4757dafc747069d0bb42fa2858601, user_id=bac58f88348a4a0fb1196afc354df351, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.589405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75781-80e7-4488-8d4b-7dcb7320b8dd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.601535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.619232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.619984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.621808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.622677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 966/1933] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:49 2022] PUT /identity/v3/projects/5db4757dafc747069d0bb42fa2858601/users/bac58f88348a4a0fb1196afc354df351/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:49.626495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.626495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.626495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.657128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.658740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.668467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] MFA Rules not processed for user `13a978dd346f45f8b2a92c9743f36f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:49.669877 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:49.680137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 968/1934] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:49.680887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.694555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac4c86-2f8b-4e95-a865-bbf0c78008df service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.820406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 967/1935] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:49.832992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.834427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.835843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.838260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.839836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.841848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.848501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.850687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:49.877756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 969/1936] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:49.879588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:49.902272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc95c2-393d-48a7-a6b3-988289e545fa admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:49.912917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:49.915752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.916388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.917694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] PATH_INFO: `/v3/projects/5db4757dafc747069d0bb42fa2858601/users/bac58f88348a4a0fb1196afc354df351/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:49.923070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db4757dafc747069d0bb42fa2858601, user_id=bac58f88348a4a0fb1196afc354df351, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:49.929159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd0b32-d269-443e-a78f-e588d5772a7b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:49.954258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:49.956579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 970/1937] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:49 2022] PUT /identity/v3/projects/5db4757dafc747069d0bb42fa2858601/users/bac58f88348a4a0fb1196afc354df351/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:49.975510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:49.975874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:49.976396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:50.025954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09f932-8668-44a9-b3e6-8b3ecd3b0965 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:50.037214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] MFA Rules not processed for user `bac58f88348a4a0fb1196afc354df351`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:50.039315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:50.098776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:50.162052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 971/1938] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:50.181632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:50.182438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:50.182737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:50.182886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:50.187050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:50.288629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:50.291008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:50.304849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 968/1939] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:50.381720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:50.383857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:50.490948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670e571-b210-4325-92b6-2f92b69db420 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:50.498622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:50.500032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:50.500582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:50.500920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] PATH_INFO: `/v3/projects/4895f8bbde1c465bbc451c6122df7dd5` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:50.502418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4895f8bbde1c465bbc451c6122df7dd5)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:50.505653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:50.570518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4895f8bbde1c465bbc451c6122df7dd5', 'request_id': 'req-042ead6b-aa23-47d2-81d7-d527e003c38a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.579639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4895f8bbde1c465bbc451c6122df7dd5', 'request_id': 'req-042ead6b-aa23-47d2-81d7-d527e003c38a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.590265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4895f8bbde1c465bbc451c6122df7dd5', 'request_id': 'req-042ead6b-aa23-47d2-81d7-d527e003c38a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.626795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] The token cache is being invalidate because project 4895f8bbde1c465bbc451c6122df7dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:50.627395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.630567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.633312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 972/1940] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:50 2022] DELETE /identity/v3/projects/4895f8bbde1c465bbc451c6122df7dd5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:50.638118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:50.655212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba17f9-36fb-4125-9dd6-c82e7ad68a01 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:50.783316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:50.784374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:50.784658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:50.784931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] PATH_INFO: `/v3/users/aaefd7ec3ee14782a3ddb50c71eff31a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:50.788660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaefd7ec3ee14782a3ddb50c71eff31a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:50.791623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:50.792415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:50.800184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:50.883260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:50.904965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaefd7ec3ee14782a3ddb50c71eff31a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.939087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaefd7ec3ee14782a3ddb50c71eff31a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:50.956943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ead6b-aa23-47d2-81d7-d527e003c38a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:50.967254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:50.968540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:50.968864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:50.969145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:50.973123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:51.013126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:51.015862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaefd7ec3ee14782a3ddb50c71eff31a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:51.018007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaefd7ec3ee14782a3ddb50c71eff31a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:51.040797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaefd7ec3ee14782a3ddb50c71eff31a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:51.057991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 969/1941] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:50 2022] DELETE /identity/v3/users/aaefd7ec3ee14782a3ddb50c71eff31a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:51.123609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:51.125570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145078f-4a25-4495-8557-65b27af01d69 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:51.137741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 973/1942] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:51.877475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:51.879992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:52.083131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6251c6-5805-45f9-99ab-57529bbfa08a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:52.088988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:52.089968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.090826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.092773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] PATH_INFO: `/v3/projects/177290e2d0e746d09b9925538669ffdf` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.092773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=177290e2d0e746d09b9925538669ffdf)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:52.108037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:52.243651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177290e2d0e746d09b9925538669ffdf', 'request_id': 'req-c864ea10-8589-4adf-855c-eade376a7555'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:52.256310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177290e2d0e746d09b9925538669ffdf', 'request_id': 'req-c864ea10-8589-4adf-855c-eade376a7555'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:52.266230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177290e2d0e746d09b9925538669ffdf', 'request_id': 'req-c864ea10-8589-4adf-855c-eade376a7555'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:52.282778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] The token cache is being invalidate because project 177290e2d0e746d09b9925538669ffdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:52.283090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:52.283974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c864ea10-8589-4adf-855c-eade376a7555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:52.289392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 970/1943] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:51 2022] DELETE /identity/v3/projects/177290e2d0e746d09b9925538669ffdf => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:52.293248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.294256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.295357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.326515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:52.328223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:52.516846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 974/1944] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:52.524646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.525395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.526063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.564380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:52.567749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:52.572323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:52.576791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:52.603045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 971/1945] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:52.608405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:52.647983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b758c-c27d-42dc-a3af-804ec4a9951b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:52.670144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:52.676467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.676467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.676467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.677340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:52.678866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:52.680978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:52.705568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba6f2-0338-47a8-a210-79fbc359e8be None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:52.712126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:52.713250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.713683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.714394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.716312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 972/1946] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:16:52.719704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:52.722973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:52.756831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9293-ebc5-4c63-ae58-4eac0be832f3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:52.770843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:52.771717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:52.772008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:52.772285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:52.773041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:52.775041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:52.886162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 973/1947] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:52 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:52.890521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:52.892962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:52.894987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:52.898442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:52.918356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 975/1948] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:52.992482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae816e-d205-40a1-9848-54c120d3ecb0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:52.999736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.000265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.000654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.001031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.001886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.004818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.041803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 974/1949] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:52 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:16:53.047725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:53.056617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:53.071334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6984d-3b90-48ab-832b-063fb9ae5fb7 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:53.081835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.082547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.082905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.083284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.084162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.085948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.128561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 976/1950] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:53.135361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:53.155737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2740161-7679-4557-81e8-82484a08c750 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:53.172933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.173736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.174123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.174514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.176352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.178560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19b420-dd83-497a-9991-0087e3504e83 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.221123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:53.241633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 975/1951] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:53.244262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161cc16-2516-4ee8-b7ab-3f864417eedb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:53.264813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.272597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.273234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.275650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] PATH_INFO: `/v3/projects/92916b71ac314c72b764347b61d6c026/users/795f7d634bec4c8382a92c706c2824fb/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.352916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92916b71ac314c72b764347b61d6c026, user_id=795f7d634bec4c8382a92c706c2824fb, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.356072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.389720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 977/1952] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:53 2022] PUT /identity/v3/projects/92916b71ac314c72b764347b61d6c026/users/795f7d634bec4c8382a92c706c2824fb/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:53.392547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345611-8171-4e4c-873a-61331fc59171 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.393091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345611-8171-4e4c-873a-61331fc59171 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.393536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74345611-8171-4e4c-873a-61331fc59171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.428886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-74345611-8171-4e4c-873a-61331fc59171 None None] MFA Rules not processed for user `795f7d634bec4c8382a92c706c2824fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:53.430531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-74345611-8171-4e4c-873a-61331fc59171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:53.591559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 976/1953] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:53.596987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:53.714744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e53aa-73c2-477e-997d-d3d3d2501b86 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:53.722739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.723962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.724257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.724574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.725477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.727651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.771544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 978/1954] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:53 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:53.776130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74345611-8171-4e4c-873a-61331fc59171 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:53.800483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74345611-8171-4e4c-873a-61331fc59171 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:53.936397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:53.938424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:53.938950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:53.939461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:53.941115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:53.944156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:53.977104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 977/1955] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:53 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:53.982979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:54.002833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0172035-54f4-4ed9-9fd7-39ebea025b7b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:54.028649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:54.030449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.031282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.032693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.039584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:54.050155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:54.107403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:54.108700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 979/1956] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:54.113986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:54.125535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b95f6-f956-44cb-a877-5fd4a431cb2f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:54.138917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:54.139886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.140169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.140445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] PATH_INFO: `/v3/projects/6e2fe074dea940e98c94762177fff1e5/users/e18093e4404c4c4ebb0c03eebd7e39dd/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.210400 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2fe074dea940e98c94762177fff1e5, user_id=e18093e4404c4c4ebb0c03eebd7e39dd, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:54.218307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:54.264063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 978/1957] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:54 2022] PUT /identity/v3/projects/6e2fe074dea940e98c94762177fff1e5/users/e18093e4404c4c4ebb0c03eebd7e39dd/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:54.266804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:54.302401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfce78-bdbc-4309-86dc-a4d9ed50be91 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:54.423044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:54.425150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:54.450261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:54.452739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.453456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.454164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.455902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:54.459843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:54.481977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 980/1958] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:54.489167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:54.510128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a50d7-54ff-4131-833e-900547b31193 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:54.519037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:54.520718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.521207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.521685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] PATH_INFO: `/v3/projects/6e2fe074dea940e98c94762177fff1e5/users/e18093e4404c4c4ebb0c03eebd7e39dd/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.525851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2fe074dea940e98c94762177fff1e5, user_id=e18093e4404c4c4ebb0c03eebd7e39dd, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:54.531314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c5505-0b26-46be-a306-466c351b6bcf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:54.553757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5267e45-3e44-433b-9ef2-e17a8f957b9d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:54.556348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 981/1959] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:54 2022] PUT /identity/v3/projects/6e2fe074dea940e98c94762177fff1e5/users/e18093e4404c4c4ebb0c03eebd7e39dd/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:54.579289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57cc8-ef64-4b01-a78e-b9ac2ed7070c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.580028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57cc8-ef64-4b01-a78e-b9ac2ed7070c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.580622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57cc8-ef64-4b01-a78e-b9ac2ed7070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.636580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8aa57cc8-ef64-4b01-a78e-b9ac2ed7070c None None] MFA Rules not processed for user `e18093e4404c4c4ebb0c03eebd7e39dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:54.639256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8aa57cc8-ef64-4b01-a78e-b9ac2ed7070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:54.655969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:54.657210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.658198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.658679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.662036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:54.669699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:54.782515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:54.784994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918a81a-0bf8-428b-a49e-17efb301aabc service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:54.818798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 982/1960] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:54.829420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 979/1961] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:54.939049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac7e6-ca0d-4313-9f36-8b69770c3742 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.940047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac7e6-ca0d-4313-9f36-8b69770c3742 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.943128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac7e6-ca0d-4313-9f36-8b69770c3742 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:54.948292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 983/1962] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:16:54 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:16:54.970025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:54.971761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:54.972556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:55.031005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:55.033013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:55.034455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ac7e6-ca0d-4313-9f36-8b69770c3742 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:55.066062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ac7e6-ca0d-4313-9f36-8b69770c3742 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:55.088560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:55.089896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:55.090479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:55.091044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:55.117556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:55.121075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:55.123047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:55.169578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 984/1963] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:16:55.216673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 980/1964] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:55.278940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:55.282897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:55.322667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299c6c2-63d2-445d-b06d-7e1d9fcc02de service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:55.333844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:55.335423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:55.336064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:55.336794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:55.355883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:55.359264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:55.402409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 985/1965] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:56.026019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:56.054491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:56.056197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:56.169505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8670eb2-6cc0-4b28-960b-96f8b2506bec None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:56.190806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:56.193027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:56.193328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:56.193740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] PATH_INFO: `/v3/users/795f7d634bec4c8382a92c706c2824fb` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:56.196437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=795f7d634bec4c8382a92c706c2824fb)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:56.206861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:56.237658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121e3aa-4fcc-4874-a7a5-176a78103b08 service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:56.262249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:56.266939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:56.267468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:56.268298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] PATH_INFO: `/v3/users/ed94855f6cf64bd69435b32d375847ad` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:56.273612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed94855f6cf64bd69435b32d375847ad)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:56.281590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:56.318667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795f7d634bec4c8382a92c706c2824fb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.348525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795f7d634bec4c8382a92c706c2824fb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.379088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795f7d634bec4c8382a92c706c2824fb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.381972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795f7d634bec4c8382a92c706c2824fb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.408132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795f7d634bec4c8382a92c706c2824fb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.426723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed94855f6cf64bd69435b32d375847ad'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.426979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 981/1966] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:56 2022] DELETE /identity/v3/users/795f7d634bec4c8382a92c706c2824fb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:56.427386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:56.431088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:56.442996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed94855f6cf64bd69435b32d375847ad'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.444512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed94855f6cf64bd69435b32d375847ad'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.484197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed94855f6cf64bd69435b32d375847ad'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.499730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cbd41017-72a6-46f1-949f-b7b996dad887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed94855f6cf64bd69435b32d375847ad'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.519353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 986/1967] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:56 2022] DELETE /identity/v3/users/ed94855f6cf64bd69435b32d375847ad => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:56.521177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778b57c-09fd-44b6-97ae-aaabab8f884b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:56.523108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778b57c-09fd-44b6-97ae-aaabab8f884b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:56.526154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778b57c-09fd-44b6-97ae-aaabab8f884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:56.542427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:56.565883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c778b57c-09fd-44b6-97ae-aaabab8f884b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:56.575496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c778b57c-09fd-44b6-97ae-aaabab8f884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:56.614669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98f835-4a85-45df-9dd5-e9973dc20315 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:56.628321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:56.735799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 987/1968] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:56.739274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:56.740054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:56.740450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:56.767752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:56.768712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:56.769022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:56.769296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] PATH_INFO: `/v3/users/85bc6b75434c4e54afcf3645a46449eb` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:56.772249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bc6b75434c4e54afcf3645a46449eb)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:56.778424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:56.781147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:56.782402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:56.806425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 988/1969] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:56.809055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:56.812668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:56.897526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bc6b75434c4e54afcf3645a46449eb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.911553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bc6b75434c4e54afcf3645a46449eb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.943629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bc6b75434c4e54afcf3645a46449eb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.944483 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bc6b75434c4e54afcf3645a46449eb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.963773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bc6b75434c4e54afcf3645a46449eb'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:56.985331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 982/1970] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:56 2022] DELETE /identity/v3/users/85bc6b75434c4e54afcf3645a46449eb => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:56.986362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:56.991050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:57.021882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5f2d0-7240-4c43-8048-d6adcdf103dc None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:57.119924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303eada-48bf-45ea-8c9c-f0c05fa8a6af admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:57.129933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:57.130850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:57.131877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:57.132618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:57.133003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:57.135246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:57.135246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:57.135246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:57.137479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:57.141236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:57.245612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:57.325901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:57.327422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:57.340521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:57.342936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:57.350121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 983/1971] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:57.352692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:57.354454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:57.377775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 989/1972] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:16:57.391023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f066f8-b95c-4e34-ae96-15a449e3cf7d service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:57.409849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:57.410767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:57.411048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:57.411324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] PATH_INFO: `/v3/projects/60a80724abf045729000dc9fb8a2440e` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:57.419153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a80724abf045729000dc9fb8a2440e)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:57.421248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:57.509836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a80724abf045729000dc9fb8a2440e', 'request_id': 'req-a3725192-4274-45f6-a67c-fce9a469b986'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.527821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a80724abf045729000dc9fb8a2440e', 'request_id': 'req-a3725192-4274-45f6-a67c-fce9a469b986'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.546204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a80724abf045729000dc9fb8a2440e', 'request_id': 'req-a3725192-4274-45f6-a67c-fce9a469b986'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.586546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] The token cache is being invalidate because project 60a80724abf045729000dc9fb8a2440e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:57.587029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.588277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.592306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 984/1973] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:57 2022] DELETE /identity/v3/projects/60a80724abf045729000dc9fb8a2440e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:57.599378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:57.611320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:57.621452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8117-8503-494a-84d1-ec56ee013906 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:57.774147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:57.776142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:57.777027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:57.777561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] PATH_INFO: `/v3/users/9bbcf6e8bc574458bd2cf57eb78818ea` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:57.779781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbcf6e8bc574458bd2cf57eb78818ea)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:57.784550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:57.892891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbcf6e8bc574458bd2cf57eb78818ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.908205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbcf6e8bc574458bd2cf57eb78818ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.910805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbcf6e8bc574458bd2cf57eb78818ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.946101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbcf6e8bc574458bd2cf57eb78818ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.963662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbcf6e8bc574458bd2cf57eb78818ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:57.979648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:57.981020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 990/1974] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:57 2022] DELETE /identity/v3/users/9bbcf6e8bc574458bd2cf57eb78818ea => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:57.986686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.012741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3725192-4274-45f6-a67c-fce9a469b986 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.113928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.114983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.115372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.115954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] PATH_INFO: `/v3/projects/3155602ef2794a7cb86556e595460b22` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.117756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3155602ef2794a7cb86556e595460b22)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:58.121726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:58.186822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:58.188581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.214428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125730c1-4fc4-4892-9b9f-ce046876c3c2 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.231289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.232468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.233351 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.233947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] PATH_INFO: `/v3/projects/92916b71ac314c72b764347b61d6c026` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.236759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92916b71ac314c72b764347b61d6c026)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:58.247459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:58.287933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3155602ef2794a7cb86556e595460b22', 'request_id': 'req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.309553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3155602ef2794a7cb86556e595460b22', 'request_id': 'req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.334660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3155602ef2794a7cb86556e595460b22', 'request_id': 'req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.334660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:58.356093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] The token cache is being invalidate because project 3155602ef2794a7cb86556e595460b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:58.356430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.357273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.363230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 985/1975] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:57 2022] DELETE /identity/v3/projects/3155602ef2794a7cb86556e595460b22 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:58.364272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:58.366413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.386916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92916b71ac314c72b764347b61d6c026', 'request_id': 'req-2194870d-6349-46e9-acd1-d5a91d3aeae7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.398568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92916b71ac314c72b764347b61d6c026', 'request_id': 'req-2194870d-6349-46e9-acd1-d5a91d3aeae7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.413764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92916b71ac314c72b764347b61d6c026', 'request_id': 'req-2194870d-6349-46e9-acd1-d5a91d3aeae7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.438829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] The token cache is being invalidate because project 92916b71ac314c72b764347b61d6c026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:58.439355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.439907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.442228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 991/1976] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:58 2022] DELETE /identity/v3/projects/92916b71ac314c72b764347b61d6c026 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:58.453224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:58.456109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.475419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:58.511719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4ecd8-24c7-46ef-b36d-9d51dcfdf251 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.564034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194870d-6349-46e9-acd1-d5a91d3aeae7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.570496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.571611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.572374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.572374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] PATH_INFO: `/v3/users/bac58f88348a4a0fb1196afc354df351` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.573903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bac58f88348a4a0fb1196afc354df351)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:58.575959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:58.592560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.593933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.594243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.594583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] PATH_INFO: `/v3/projects/33fd54359bb44abb83c5885aa27b0509` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.595990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fd54359bb44abb83c5885aa27b0509)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:58.598173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:58.641839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bac58f88348a4a0fb1196afc354df351'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.660603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bac58f88348a4a0fb1196afc354df351'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.661795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bac58f88348a4a0fb1196afc354df351'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.666783 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fd54359bb44abb83c5885aa27b0509', 'request_id': 'req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.676295 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fd54359bb44abb83c5885aa27b0509', 'request_id': 'req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.684141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bac58f88348a4a0fb1196afc354df351'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.684471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fd54359bb44abb83c5885aa27b0509', 'request_id': 'req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.694397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bac58f88348a4a0fb1196afc354df351'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.702638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 992/1977] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:58 2022] DELETE /identity/v3/users/bac58f88348a4a0fb1196afc354df351 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:58.704225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:58.704840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] The token cache is being invalidate because project 33fd54359bb44abb83c5885aa27b0509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:58.705318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.706492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2bcafd16-f99e-432a-b9a8-e9b6cebf9a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.709478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.710129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 986/1978] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:58 2022] DELETE /identity/v3/projects/33fd54359bb44abb83c5885aa27b0509 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:58.721785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fdc79-b7ff-4a37-b0e1-4831a71f1cc0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.722030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fdc79-b7ff-4a37-b0e1-4831a71f1cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.722570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fdc79-b7ff-4a37-b0e1-4831a71f1cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.757092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-827fdc79-b7ff-4a37-b0e1-4831a71f1cc0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:58.759204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-827fdc79-b7ff-4a37-b0e1-4831a71f1cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.796883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:58.843963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcb9bd-686e-4efd-8455-086cad0ee9a1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.852159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.853643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.853973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.854294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] PATH_INFO: `/v3/users/e18093e4404c4c4ebb0c03eebd7e39dd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.856733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18093e4404c4c4ebb0c03eebd7e39dd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:58.859386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:58.891218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 987/1979] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:16:58.893946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.894462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.896696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:58.926398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:58.928285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:58.933352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:58.950832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 988/1980] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:58.959898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:58.983934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7dbbeb-031e-4465-be2b-f82baed44f78 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:58.992313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18093e4404c4c4ebb0c03eebd7e39dd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:58.995196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:58.997150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:58.997807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:58.998962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.000815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:16:59.001430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.006817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.009948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18093e4404c4c4ebb0c03eebd7e39dd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.011354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18093e4404c4c4ebb0c03eebd7e39dd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.043903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 989/1981] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:16:59.048880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.051985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18093e4404c4c4ebb0c03eebd7e39dd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.070788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a809-ff1a-4011-9c07-8a3d4bdd0ec8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.076081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18093e4404c4c4ebb0c03eebd7e39dd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.079325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.080401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.081315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.081868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.082824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.085430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.092347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 993/1982] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:58 2022] DELETE /identity/v3/users/e18093e4404c4c4ebb0c03eebd7e39dd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:59.119928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 990/1983] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:16:59 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:16:59.127019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.129491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.237865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.242186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.310245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c8e9b-da8c-4929-b92a-e2124b7a0dd7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.375358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb15ca-97de-4fc0-a217-a301c454e136 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.382567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.383357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.383742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.384329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] PATH_INFO: `/v3/projects/5db4757dafc747069d0bb42fa2858601` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.385862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db4757dafc747069d0bb42fa2858601)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.387061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.387984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.388255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.388673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.389396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.391726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.393366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.428281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 994/1984] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:16:59 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:16:59.433294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.447354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290874-70b4-4065-ba10-3d3f3d1b05ec admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.473627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db4757dafc747069d0bb42fa2858601', 'request_id': 'req-d1ad32ef-8560-4294-a821-50d89fadccd0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.488341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db4757dafc747069d0bb42fa2858601', 'request_id': 'req-d1ad32ef-8560-4294-a821-50d89fadccd0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.501435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db4757dafc747069d0bb42fa2858601', 'request_id': 'req-d1ad32ef-8560-4294-a821-50d89fadccd0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.515793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] The token cache is being invalidate because project 5db4757dafc747069d0bb42fa2858601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:59.516317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.517903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d1ad32ef-8560-4294-a821-50d89fadccd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.520193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 991/1985] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:59 2022] DELETE /identity/v3/projects/5db4757dafc747069d0bb42fa2858601 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:16:59.524116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:59.527354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.527957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.528396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.554693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:59.556131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.568182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:59.574704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.576099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.576690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.577257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.578641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.594060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.616496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 995/1986] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:16:59.617685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.622105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.641153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64116fd6-93a4-4abb-92ed-2f07af4baa6a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.648436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.649838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.650310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.650765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] PATH_INFO: `/v3/projects/6e2fe074dea940e98c94762177fff1e5` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.652531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 992/1987] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:59.652844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2fe074dea940e98c94762177fff1e5)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.653755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.655805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.656417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.677145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379677f-b4f8-430b-9c30-bd440506ca63 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.684728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.685924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.686413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.686887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.690230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.691364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97f5c-3c7e-4df2-8628-4b36679b9e59 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.712503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 993/1988] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:16:59.712503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.712503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.712503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.741644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:59.742938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.749353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2fe074dea940e98c94762177fff1e5', 'request_id': 'req-9c627028-e122-4aff-9597-f4fe46f49c00'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.756709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2fe074dea940e98c94762177fff1e5', 'request_id': 'req-9c627028-e122-4aff-9597-f4fe46f49c00'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.763252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2fe074dea940e98c94762177fff1e5', 'request_id': 'req-9c627028-e122-4aff-9597-f4fe46f49c00'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.779329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] The token cache is being invalidate because project 6e2fe074dea940e98c94762177fff1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:16:59.779721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.780406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.782110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 996/1989] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:16:59 2022] DELETE /identity/v3/projects/6e2fe074dea940e98c94762177fff1e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:59.784325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.785956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.822260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 994/1990] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:16:59.823589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.825452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.838567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2759aa4-4c11-43dd-a0dd-cf99efb681fe None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.843808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.844910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.845332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.845654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] PATH_INFO: `/v3/users/19c2c9e691d2482a906c230096e56bae` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.847793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c2c9e691d2482a906c230096e56bae)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.850667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.862521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c627028-e122-4aff-9597-f4fe46f49c00 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:16:59.867334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:16:59.868278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.868566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.868841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] PATH_INFO: `/v3/projects/6d6b10c6fa96449ea3579707919671cf/users/53c746c9a15b4429804423cd1e234ec8/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.887922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6b10c6fa96449ea3579707919671cf, user_id=53c746c9a15b4429804423cd1e234ec8, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:16:59.890943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291b2ab-86fd-418c-a6de-78cac37ac875 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:16:59.913719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 997/1991] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:16:59 2022] PUT /identity/v3/projects/6d6b10c6fa96449ea3579707919671cf/users/53c746c9a15b4429804423cd1e234ec8/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:59.916056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d62d-dbd1-48c2-876b-5488d3775a4e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:16:59.916536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d62d-dbd1-48c2-876b-5488d3775a4e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:16:59.917008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d62d-dbd1-48c2-876b-5488d3775a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:16:59.919465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c2c9e691d2482a906c230096e56bae'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.933532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c2c9e691d2482a906c230096e56bae'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.944324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ef83d62d-dbd1-48c2-876b-5488d3775a4e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:16:59.945655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ef83d62d-dbd1-48c2-876b-5488d3775a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.949384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c2c9e691d2482a906c230096e56bae'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.950200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c2c9e691d2482a906c230096e56bae'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.958023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c2c9e691d2482a906c230096e56bae'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:16:59.965585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 995/1992] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:16:59 2022] DELETE /identity/v3/users/19c2c9e691d2482a906c230096e56bae => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:16:59.966440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:16:59.968016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:16:59.994204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:16:59.999660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:00.026496 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 998/1993] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:00.028134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2161-b423-4075-b23a-37384ec38086 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.028534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2161-b423-4075-b23a-37384ec38086 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.028880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2161-b423-4075-b23a-37384ec38086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.068989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-579f2161-b423-4075-b23a-37384ec38086 None None] MFA Rules not processed for user `53c746c9a15b4429804423cd1e234ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:00.070014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f41d42-da54-4176-b66a-b03f2b1c7873 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.070867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-579f2161-b423-4075-b23a-37384ec38086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.077892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.078764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.079220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.079717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.080841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:00.081577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.084452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399ae47-1308-4003-9880-e4e7c4287da3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.101005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 996/1994] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:00.102375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8163886-d929-46ff-b631-0be31957db8f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.102665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8163886-d929-46ff-b631-0be31957db8f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.102943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8163886-d929-46ff-b631-0be31957db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.125968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c8163886-d929-46ff-b631-0be31957db8f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:00.127284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c8163886-d929-46ff-b631-0be31957db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.142491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 997/1995] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:00.142491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.142491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.142491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.173982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:00.175233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.183892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 998/1996] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:00.184786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.192722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 999/1997] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:00.192722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2161-b423-4075-b23a-37384ec38086 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.192722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-579f2161-b423-4075-b23a-37384ec38086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.200247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846063b-0665-4343-a2bc-ce4c821796cc None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.205672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.206364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.206716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.207055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.207817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.209823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.238309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 999/1998] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:00.239365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.289985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2161-b423-4075-b23a-37384ec38086 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.322850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37bf48-515e-4dc6-b538-afefc460bc40 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.328215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.328918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.329201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.329490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.330215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:00.330532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.332220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.351776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1000/1999] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:00.351776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.354433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.367709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.368358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.368358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.368549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.388457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.391056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.408413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1000/2000] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:00.409781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.429051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40c5a9-d7b5-4822-b2dc-524c2b702356 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.437023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.438283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.438769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.439772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.441184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.444096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.467949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44345b8-4f4e-47d6-800f-272e54cdc6ae admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.471685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1001/2001] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/users => generated 294 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:00.472355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.475341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.476775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.477367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.477913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.483371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70963-e967-4576-9dcd-bb601d885dce admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.489356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.490729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.490729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.491132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.491755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.494034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.509999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:00.532528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1002/2002] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:00.533592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.551900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:00.621522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19a702-67af-4930-8687-9a4964931b0f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.626418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.627624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.627624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.627974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.628465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.630135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.635773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.639026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.641553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1003/2003] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:00.642479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.652486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1001/2004] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:00.653418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.654584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a6dc7-02aa-457e-b78e-83635ca06bb5 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.660176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.660506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.660670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.661745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.661745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.663704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.670672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c296d-4ed9-4dfe-b9b9-9eb9b79239ea service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.685743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.686458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.686778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.687055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.687900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.689622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.694748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1004/2005] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:00.699540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.706355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1002/2006] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:00.710678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.714715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65301558-2426-43b7-accf-5c962ec5214b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.720432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.721253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.721550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.721810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.722569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.724402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.724540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92495cc8-43e0-4ed9-9155-1f3401dfe338 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.730911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.732064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.732420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.732768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] PATH_INFO: `/v3/projects/0c07e96add0a41258a66c7c7442b1c95/users/e4e720331fb643e986221cbe62e4d4c1/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.739990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1005/2007] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:00.744107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.757902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe5ec9-7adf-4969-afaf-7888e43fe9cf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.759924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c07e96add0a41258a66c7c7442b1c95, user_id=e4e720331fb643e986221cbe62e4d4c1, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.763013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4173a-2278-4fb9-837e-b90e0d0475a6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.769449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.770563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.771104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.771709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.773769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.776654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.782025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1003/2008] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:00 2022] PUT /identity/v3/projects/0c07e96add0a41258a66c7c7442b1c95/users/e4e720331fb643e986221cbe62e4d4c1/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:00.792026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1006/2009] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:00.794114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056fa0-3c5a-433d-b2a7-0d3abd64e2ee None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.794754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056fa0-3c5a-433d-b2a7-0d3abd64e2ee None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.795212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056fa0-3c5a-433d-b2a7-0d3abd64e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.797969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.814803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7575ea-5546-42f4-b001-fb1e1da8a76e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:00.844956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1c056fa0-3c5a-433d-b2a7-0d3abd64e2ee None None] MFA Rules not processed for user `e4e720331fb643e986221cbe62e4d4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:00.846830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1c056fa0-3c5a-433d-b2a7-0d3abd64e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:00.925673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:00.928577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:00.929004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:00.929526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] PATH_INFO: `/v3/projects/c36ce6d9ddaa46c6b30e782ae96733f6/users/b47454271e4d48849eda8d7408ec84e5/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:00.952405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36ce6d9ddaa46c6b30e782ae96733f6, user_id=b47454271e4d48849eda8d7408ec84e5, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:00.955501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:00.979606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1007/2010] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:00 2022] PUT /identity/v3/projects/c36ce6d9ddaa46c6b30e782ae96733f6/users/b47454271e4d48849eda8d7408ec84e5/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:00.981077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:00.986543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.008080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1004/2011] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:01.008080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.008080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.008080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.041614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] MFA Rules not processed for user `b47454271e4d48849eda8d7408ec84e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:01.042990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.059626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:01.126437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10676e-2595-4087-a0f1-f04aedeeae48 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.127384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1005/2012] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:01.128636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.140825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619bdd1-9f80-45b2-b81b-2afd961d3cf8 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.154522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.156734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.157085 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.157411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.158198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.160374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.161252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.161422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.161564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.161894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] PATH_INFO: `/v3/projects/1decfa476faf415581dacad34b31320c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.163218 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1decfa476faf415581dacad34b31320c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.165263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.171466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:01.226902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1006/2013] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:01 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:01.226902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.237025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1decfa476faf415581dacad34b31320c', 'request_id': 'req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.245707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1decfa476faf415581dacad34b31320c', 'request_id': 'req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.253678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1decfa476faf415581dacad34b31320c', 'request_id': 'req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.271222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] The token cache is being invalidate because project 1decfa476faf415581dacad34b31320c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:01.271546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.272254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.273888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1008/2014] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:00 2022] DELETE /identity/v3/projects/1decfa476faf415581dacad34b31320c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:01.274753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.276299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.312159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0dc6c-e8f4-4fd4-aa16-110843938308 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.319276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.320497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.321190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.322061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.325457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.363248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0365d-a889-4dca-8fa1-5d0f958bafbd admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.418221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.420830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.435191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.435955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.436196 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.436482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.437234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.437399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1007/2015] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:01.438957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.441217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.446035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.468341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fcb67b-76a4-494e-81ac-c4b1d1c61125 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.468932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:01.480778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1009/2016] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:01 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:01.481053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.481232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.481410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.481410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.481410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] PATH_INFO: `/v3/users/eacae3c80f7744a7b3da09f0f991ef94` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.482678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.483990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eacae3c80f7744a7b3da09f0f991ef94)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.487844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.496246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19648fa0-1424-4b8d-8f49-6707f60901c7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.502535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.503437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.503767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.504197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] PATH_INFO: `/v3/users/b47454271e4d48849eda8d7408ec84e5` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.505594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b47454271e4d48849eda8d7408ec84e5)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.508086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.576549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eacae3c80f7744a7b3da09f0f991ef94'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.581019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b47454271e4d48849eda8d7408ec84e5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.591080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eacae3c80f7744a7b3da09f0f991ef94'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.595920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b47454271e4d48849eda8d7408ec84e5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.597448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b47454271e4d48849eda8d7408ec84e5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.622140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eacae3c80f7744a7b3da09f0f991ef94'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.623864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eacae3c80f7744a7b3da09f0f991ef94'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.634500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b47454271e4d48849eda8d7408ec84e5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.640846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eacae3c80f7744a7b3da09f0f991ef94'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.654982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-caa83f2d-3993-4b4c-8911-4578d22aabfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b47454271e4d48849eda8d7408ec84e5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:01.656237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1008/2017] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:01 2022] DELETE /identity/v3/users/eacae3c80f7744a7b3da09f0f991ef94 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:01.662840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.665204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.694825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1010/2018] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:01 2022] DELETE /identity/v3/users/b47454271e4d48849eda8d7408ec84e5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:01.704075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:01.714745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.716583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.717161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.788565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:01.791233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.814971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74625239-84a8-4c51-a136-6fd1feb64c0c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:01.908897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1011/2019] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:01.920544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.922209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.932163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:01.933265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:01.933641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:01.934465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:01.935373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:01.937189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:01.951718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1009/2020] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:01.958076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:01.960577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:01.989512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e256df-c4ce-4ca8-add8-8ee6af86c444 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.000897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.002315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.002619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.002902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] PATH_INFO: `/v3/projects/2edc870762ed445b844d076b6cbcf2f8/users/67a304cd570448079213979b753c4651/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.030061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2edc870762ed445b844d076b6cbcf2f8, user_id=67a304cd570448079213979b753c4651, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.034024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95435d-e46f-40cf-843d-51dd6bd3b2a5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.035872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.039948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.040732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.043132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.043652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.056385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.057632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.064644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1010/2021] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:01 2022] PUT /identity/v3/projects/2edc870762ed445b844d076b6cbcf2f8/users/67a304cd570448079213979b753c4651/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:02.069589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1012/2022] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:02.070928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.173504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb06dd7-0359-437b-874d-93a5472ec807 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.186688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.187473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.188171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.189058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.191463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.193578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d8b33-b2b9-4828-91d0-dc27b00a9f29 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.209989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1011/2023] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:02.212225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.235024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb74b7-d00e-4059-9fbc-3946c13c53ee service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.240668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.242355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.242652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.243266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] PATH_INFO: `/v3/projects/2edc870762ed445b844d076b6cbcf2f8/users/67a304cd570448079213979b753c4651/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.245946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2edc870762ed445b844d076b6cbcf2f8, user_id=67a304cd570448079213979b753c4651, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.248811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.287684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1013/2024] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:02 2022] PUT /identity/v3/projects/2edc870762ed445b844d076b6cbcf2f8/users/67a304cd570448079213979b753c4651/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:02.294075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.294075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.294405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.352796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.355111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.361571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:02.369722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] MFA Rules not processed for user `67a304cd570448079213979b753c4651`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:02.379126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.436232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:02.544619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5af70-7002-4877-9619-74e27e36c4bb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.545715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1012/2025] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:02.554613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.555573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.555860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.556167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] PATH_INFO: `/v3/projects/1d56f71e51c64a07badb794c54ffaf97` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.560664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.563181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.570041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d56f71e51c64a07badb794c54ffaf97)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.572166 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.665700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d56f71e51c64a07badb794c54ffaf97', 'request_id': 'req-e52f36ff-58b7-4b9a-afa9-d2801653e709'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.684781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ac95c-5e0b-438e-9773-b239cfac8909 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.685244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d56f71e51c64a07badb794c54ffaf97', 'request_id': 'req-e52f36ff-58b7-4b9a-afa9-d2801653e709'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.696202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d56f71e51c64a07badb794c54ffaf97', 'request_id': 'req-e52f36ff-58b7-4b9a-afa9-d2801653e709'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.716635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] The token cache is being invalidate because project 1d56f71e51c64a07badb794c54ffaf97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:02.717016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.717985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.721284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1014/2026] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:02 2022] DELETE /identity/v3/projects/1d56f71e51c64a07badb794c54ffaf97 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:02.722597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.725018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.750195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.751689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.752234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.752714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.756410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.828734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f36ff-58b7-4b9a-afa9-d2801653e709 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.835519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:02.836910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:02.837435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:02.837642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] PATH_INFO: `/v3/projects/c36ce6d9ddaa46c6b30e782ae96733f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:02.839154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36ce6d9ddaa46c6b30e782ae96733f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.841805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.888751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:02.891822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:02.899136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:02.916666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1013/2027] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:02.921440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:02.923456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:02.930260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36ce6d9ddaa46c6b30e782ae96733f6', 'request_id': 'req-5aaaaefa-3f3e-4f1c-919c-620db67bd822'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.938489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36ce6d9ddaa46c6b30e782ae96733f6', 'request_id': 'req-5aaaaefa-3f3e-4f1c-919c-620db67bd822'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.948167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36ce6d9ddaa46c6b30e782ae96733f6', 'request_id': 'req-5aaaaefa-3f3e-4f1c-919c-620db67bd822'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.956151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b34a7-3944-456c-b7d8-29d32eed04d2 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:02.989090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] The token cache is being invalidate because project c36ce6d9ddaa46c6b30e782ae96733f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:02.989585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.990788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aaaaefa-3f3e-4f1c-919c-620db67bd822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:02.993859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1015/2028] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:02 2022] DELETE /identity/v3/projects/c36ce6d9ddaa46c6b30e782ae96733f6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:03.002311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.002819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.003311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.010772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:03.034342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:03.035921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.104266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.106620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.107123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.107631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] PATH_INFO: `/v3/users/13a978dd346f45f8b2a92c9743f36f8f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.110072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a978dd346f45f8b2a92c9743f36f8f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.115010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.117631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1016/2029] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:03.118722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.120663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.140468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3259a-5173-41bc-9648-be29dda01936 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.146981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.148204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.148649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.149274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] PATH_INFO: `/v3/users/e4e720331fb643e986221cbe62e4d4c1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.152041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e720331fb643e986221cbe62e4d4c1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.156543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.267306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e720331fb643e986221cbe62e4d4c1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.271997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a978dd346f45f8b2a92c9743f36f8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.276323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e720331fb643e986221cbe62e4d4c1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.277391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e720331fb643e986221cbe62e4d4c1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.284956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a978dd346f45f8b2a92c9743f36f8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.293610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e720331fb643e986221cbe62e4d4c1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.305122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-70ff1110-a987-414f-94a1-c4690a435391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e720331fb643e986221cbe62e4d4c1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.306456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a978dd346f45f8b2a92c9743f36f8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.307170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a978dd346f45f8b2a92c9743f36f8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.312969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1017/2030] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:03 2022] DELETE /identity/v3/users/e4e720331fb643e986221cbe62e4d4c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:03.316168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2eb8e71f-2716-4fd5-9303-d4250e132f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a978dd346f45f8b2a92c9743f36f8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.323827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.324133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.324523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.325198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1014/2031] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:02 2022] DELETE /identity/v3/users/13a978dd346f45f8b2a92c9743f36f8f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:03.327341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.327931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.328417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.354002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:03.357300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.360599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:03.361848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.432346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:03.473058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1015/2032] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:03.487392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.488900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.490247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1018/2033] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:03.497034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.514484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33069165-3136-4794-8d8c-4090342e5d58 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.523919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.525134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.525640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.526140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.527769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:03.528332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.532084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.548624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:03.559824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1019/2034] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:17:03.562693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.570156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f2c69-411c-46ed-b063-f3de437986aa None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.576190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.576893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.577182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.577347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c0325-87d6-4bad-8a31-16afff247a06 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.577516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.583582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.584295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.584888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.585169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.586202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.588269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.590042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.592052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.605810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1016/2035] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:03.621609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1020/2036] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:03 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:03.627562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.646875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2606561-053f-4b19-9f48-fdd57ae0c99e service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.668987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.672271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.730338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.730338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.730726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.731007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.731747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.733973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.758161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1017/2037] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:03 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:03.762554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.775207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ff5f-3d0c-4eb9-91c6-3e5de730202c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.781400 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.782480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.783068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.783215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.784826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.787122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.789514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1ae32-3968-40b1-8379-0abd8e87a93a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.797932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.799717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.800215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.803459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] PATH_INFO: `/v3/users/53c746c9a15b4429804423cd1e234ec8` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.803627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1018/2038] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:03.803747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c746c9a15b4429804423cd1e234ec8)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.805093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.808055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.825114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be62bf-1000-4689-bef0-173178179e82 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.836487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.837518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.837919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.838482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.840420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.843940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.867235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1019/2039] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:03.868570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.884585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce181f-edd1-4f87-add3-09acd1ca22aa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:03.891215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:03.892464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.892850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.893242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] PATH_INFO: `/v3/projects/ae3b770f0ffa4fb58a6c5d4f0d57bc0b/users/f4ec145ed78f4a3990cb2d97555c1987/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:03.907944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c746c9a15b4429804423cd1e234ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.917362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3b770f0ffa4fb58a6c5d4f0d57bc0b, user_id=f4ec145ed78f4a3990cb2d97555c1987, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:03.921806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:03.923181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c746c9a15b4429804423cd1e234ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.924419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c746c9a15b4429804423cd1e234ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.941981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1020/2040] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:03 2022] PUT /identity/v3/projects/ae3b770f0ffa4fb58a6c5d4f0d57bc0b/users/f4ec145ed78f4a3990cb2d97555c1987/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:03.941981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:03.954288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:03.956461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c746c9a15b4429804423cd1e234ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.974527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2570ee2d-3ca8-417f-b81f-1d4e01a12f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c746c9a15b4429804423cd1e234ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:03.987540 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1021/2041] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:03 2022] DELETE /identity/v3/users/53c746c9a15b4429804423cd1e234ec8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:03.988859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb139c-ff44-4476-97f1-b256b0ae328d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:03.989136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb139c-ff44-4476-97f1-b256b0ae328d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:03.989417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb139c-ff44-4476-97f1-b256b0ae328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.027761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1bfb139c-ff44-4476-97f1-b256b0ae328d None None] MFA Rules not processed for user `f4ec145ed78f4a3990cb2d97555c1987`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:04.032301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1bfb139c-ff44-4476-97f1-b256b0ae328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.047417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:04.057351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0769e-85f6-4440-8fb2-1d737ad33c31 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.133473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:04.160332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1022/2042] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:04.161453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.161704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.162153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.188041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.192785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.193237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.193874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] PATH_INFO: `/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.199206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf03ca75d32149be8b6d6577f4bb08c7)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:04.202488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:04.203851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.208705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:04.212838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1023/2043] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:04.213711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:04.215115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.306203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf03ca75d32149be8b6d6577f4bb08c7', 'request_id': 'req-a42404eb-c243-4499-92a6-7188e2c8042f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.316887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16070d3b-3b91-4276-b8f6-ec6b9656e77d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.318566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf03ca75d32149be8b6d6577f4bb08c7', 'request_id': 'req-a42404eb-c243-4499-92a6-7188e2c8042f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.328025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf03ca75d32149be8b6d6577f4bb08c7', 'request_id': 'req-a42404eb-c243-4499-92a6-7188e2c8042f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.341355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] The token cache is being invalidate because project bf03ca75d32149be8b6d6577f4bb08c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:04.341650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.342534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.344538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1021/2044] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:03 2022] DELETE /identity/v3/projects/bf03ca75d32149be8b6d6577f4bb08c7 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:04.348869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:04.351069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.418493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.419415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.419767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.420041 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.424635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.442600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42404eb-c243-4499-92a6-7188e2c8042f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.451285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.453302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.453864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.454300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] PATH_INFO: `/v3/projects/0c07e96add0a41258a66c7c7442b1c95` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.456747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c07e96add0a41258a66c7c7442b1c95)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:04.461059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:04.535191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:04.538010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:04.538358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c07e96add0a41258a66c7c7442b1c95', 'request_id': 'req-3138525d-b9e0-4ccd-9bbe-f0430577b54d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.548580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c07e96add0a41258a66c7c7442b1c95', 'request_id': 'req-3138525d-b9e0-4ccd-9bbe-f0430577b54d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.558470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c07e96add0a41258a66c7c7442b1c95', 'request_id': 'req-3138525d-b9e0-4ccd-9bbe-f0430577b54d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.565442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:04.566361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1024/2045] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:04.570158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:04.574255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.584344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] The token cache is being invalidate because project 0c07e96add0a41258a66c7c7442b1c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:04.584773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.585826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3138525d-b9e0-4ccd-9bbe-f0430577b54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.592612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1022/2046] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:04 2022] DELETE /identity/v3/projects/0c07e96add0a41258a66c7c7442b1c95 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:04.592612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.592612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.592612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.618282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:04.620100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.643269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:04.729157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82641c9f-09fa-4b3f-89ba-5e282f13b296 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.769733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1023/2047] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:04.771705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:04.773869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.800975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7913d-b328-4ffd-9b61-d18c82ee2654 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.806402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.808139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.808671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.809229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] PATH_INFO: `/v3/users/67a304cd570448079213979b753c4651` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.809476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.810267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.810554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.810834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.811326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a304cd570448079213979b753c4651)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:04.813154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.814771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:04.829087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:04.831091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b378-0ad6-4e41-924c-17eafc5f2a06 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:04.859569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1025/2048] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:04.861480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:04.861796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:04.862128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:04.900144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a304cd570448079213979b753c4651'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.909739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:04.910895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:04.918091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a304cd570448079213979b753c4651'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.944415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1026/2049] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:04.950453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:04.954963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a304cd570448079213979b753c4651'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.956128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a304cd570448079213979b753c4651'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.970861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b877-6790-4ad8-b9a1-c4870c18e494 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:04.976870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a304cd570448079213979b753c4651'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:04.993075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1024/2050] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:04 2022] DELETE /identity/v3/users/67a304cd570448079213979b753c4651 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:04.996337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:04.999348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.000034 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.004312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.006581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:05.007740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.012976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.057901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1027/2051] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:05.062647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.064297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:05.165463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b4704-68d5-4388-affb-17717213df4d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.183366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.184191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.184537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.184885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.185706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.188031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.217389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:05.250914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1025/2052] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:05 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:05.256489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.270592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974830-d153-45d1-8389-b684d7b37ea7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.398393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.399301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.399613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.399893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.400626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.402734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.429743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1028/2053] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:05 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:05.434325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.446899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750aaae-9901-4262-81b1-7cb8c1d1c035 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.452334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.453042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.453328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.454414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.454619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.456273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.470199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1026/2054] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:05.474799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.493408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a774-3739-4e97-94ae-2a7917be6a0d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.501997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.503161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.503705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.504190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.505620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.508946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.529668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1029/2055] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:05.536255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.543228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.547632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:05.551863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee3cb4-60f3-4d74-8b9e-31fb17b57d21 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.557411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.558372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.558656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.558932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] PATH_INFO: `/v3/projects/994738157bca4d9d887f316699bd9387/users/cf7a76ad651d4f7996116c942103a496/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.574321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680032a9-0e75-416e-9e13-58bc89bc06cf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.583372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.585236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.585607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.586065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] PATH_INFO: `/v3/projects/6d6b10c6fa96449ea3579707919671cf` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.589297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6b10c6fa96449ea3579707919671cf)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.594279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.607105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=994738157bca4d9d887f316699bd9387, user_id=cf7a76ad651d4f7996116c942103a496, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.611138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.646084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1027/2056] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:05 2022] PUT /identity/v3/projects/994738157bca4d9d887f316699bd9387/users/cf7a76ad651d4f7996116c942103a496/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:05.646937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.653983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:05.656441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:05.697669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6b10c6fa96449ea3579707919671cf', 'request_id': 'req-3ffb571b-d671-4684-a191-74cf4fb60177'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:05.710189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6b10c6fa96449ea3579707919671cf', 'request_id': 'req-3ffb571b-d671-4684-a191-74cf4fb60177'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:05.719355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6b10c6fa96449ea3579707919671cf', 'request_id': 'req-3ffb571b-d671-4684-a191-74cf4fb60177'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:05.744478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] The token cache is being invalidate because project 6d6b10c6fa96449ea3579707919671cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:05.744952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:05.746095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ffb571b-d671-4684-a191-74cf4fb60177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:05.749352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1030/2057] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:05 2022] DELETE /identity/v3/projects/6d6b10c6fa96449ea3579707919671cf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:05.751459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916fa5f-6e92-47c7-b6c7-06207ad82dc5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.752001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916fa5f-6e92-47c7-b6c7-06207ad82dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.752479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916fa5f-6e92-47c7-b6c7-06207ad82dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.771246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442b5ff-5fdd-437e-ae13-123f459bb5ae admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:05.780549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7916fa5f-6e92-47c7-b6c7-06207ad82dc5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:05.781912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7916fa5f-6e92-47c7-b6c7-06207ad82dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:05.909675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1031/2058] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:05.909675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.909675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.909675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.930509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:05.931566 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:05.931927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:05.932274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] PATH_INFO: `/v3/projects/2edc870762ed445b844d076b6cbcf2f8` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:05.933817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2edc870762ed445b844d076b6cbcf2f8)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:05.939875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:05.953993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:05.956373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:05.970922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1032/2059] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:05.972462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:05.978217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:06.011949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2edc870762ed445b844d076b6cbcf2f8', 'request_id': 'req-28a495d6-a4b1-4705-9377-0c1a51d51650'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:06.024577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2edc870762ed445b844d076b6cbcf2f8', 'request_id': 'req-28a495d6-a4b1-4705-9377-0c1a51d51650'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:06.034199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2edc870762ed445b844d076b6cbcf2f8', 'request_id': 'req-28a495d6-a4b1-4705-9377-0c1a51d51650'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:06.040873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:06.057730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] The token cache is being invalidate because project 2edc870762ed445b844d076b6cbcf2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:06.058058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:06.059297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-28a495d6-a4b1-4705-9377-0c1a51d51650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:06.061564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1028/2060] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:05 2022] DELETE /identity/v3/projects/2edc870762ed445b844d076b6cbcf2f8 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:06.063812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89331-09e0-4506-9ff1-8d57dc9370c8 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.064524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89331-09e0-4506-9ff1-8d57dc9370c8 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.064802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd89331-09e0-4506-9ff1-8d57dc9370c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.081192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dc1d1-6989-450d-9db7-aad557187109 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.088873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5bd89331-09e0-4506-9ff1-8d57dc9370c8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:06.090250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5bd89331-09e0-4506-9ff1-8d57dc9370c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:06.153730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.154494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.155065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.155364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.162143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:06.194169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:06.195672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1029/2061] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:06.197253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.197539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.197814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.244692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:06.246098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:06.291056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.292428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1030/2062] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:06.294405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd2e83-e371-4160-91fb-87e93886a0a5 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.305611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.321820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1033/2063] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:06.326350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.326896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.327352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.328964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04da7d5-c210-4293-a6ef-fbf82fce8c24 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.342236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.344183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.344771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.345360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.347732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:06.348431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.354706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.372741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:06.376715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:06.382271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1031/2064] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:06.388362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.403008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1034/2065] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:06.410109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.430529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dc374-c6b8-4fa5-943f-34948f86134c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.432332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651ae-5e88-496e-84e1-f1d11bf7e770 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.442449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.444178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.445524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.446281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.448087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:06.449117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.449696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.450875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.451248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.451694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.452535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.454005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.455255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.489557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1035/2066] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:06.497986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.535363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1032/2067] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:06.536382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.536842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b98f2-1286-4501-858b-f0767f59d439 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.683866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.684796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.685734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.685734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.686692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.689766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.691072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57ab3-47ad-421d-905f-385942ac5ca7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.702382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.703731 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.704242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.704885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.706113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.708770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.739038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1036/2068] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/projects => generated 409 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:06.740210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.759695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1033/2069] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:06.764963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.788684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62732e5-64ba-46e5-b020-3c0cde897839 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.854546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001a950-797c-4576-97f8-1b2392b922db admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.862616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.863858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.864404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.864829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.865788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.868598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.904121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1037/2070] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:06 2022] POST /identity/v3/users => generated 305 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:06.905171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.916571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:06.917882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ebf6-344d-402a-aac1-6c7277a19922 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.925185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.926226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.926377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.926894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.927906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.929693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.933589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:06.955425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1038/2071] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:06.956720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:06.959675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.961045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.961838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.962426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.964168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.967782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:06.970577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d56f1-97be-4fc6-aab4-a78838951a0b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:06.993525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:06.994939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:06.995465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:06.996149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:06.997120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:06.997635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1034/2072] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:07.002060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.002967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.030473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945da37-655b-45d7-9609-39a072ca3830 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.035083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1039/2073] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:07.037308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.043368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.045881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.046413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.046838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.048620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.052356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.065985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f28a-84c1-4136-a6e9-deebfb975692 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.076763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1035/2074] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:07.077979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.082788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.084863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.085468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.086065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] PATH_INFO: `/v3/projects/fff51e7d46e14bdfa40052adb150384e/users/3c867d69f3af4cb4abd035e9a058b686/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.090388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f8eea-9f55-46ea-94f6-c775866934c5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.099229 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.100928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.101217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.101789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] PATH_INFO: `/v3/projects/0cae41d6b94a4850b9e699be985a295f/users/caa581494f95446ba423c136c378d2e3/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.114636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff51e7d46e14bdfa40052adb150384e, user_id=3c867d69f3af4cb4abd035e9a058b686, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.117875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9609c43-3353-4c48-9706-ec96397b1ec7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.123830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae41d6b94a4850b9e699be985a295f, user_id=caa581494f95446ba423c136c378d2e3, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.126387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe5ab8-df1b-49d7-b114-6e24df402e52 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.145142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1040/2075] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:07 2022] PUT /identity/v3/projects/fff51e7d46e14bdfa40052adb150384e/users/3c867d69f3af4cb4abd035e9a058b686/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:07.155736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1036/2076] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:07 2022] PUT /identity/v3/projects/0cae41d6b94a4850b9e699be985a295f/users/caa581494f95446ba423c136c378d2e3/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:07.157903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.159098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.159683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.162986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.164106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.164417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.201645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] MFA Rules not processed for user `3c867d69f3af4cb4abd035e9a058b686`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:07.203223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:07.205415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] MFA Rules not processed for user `caa581494f95446ba423c136c378d2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:07.206609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:07.295355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1037/2077] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:07.328066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.329678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:07.368938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1041/2078] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:07.370350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.415207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659ca7-814a-40d0-bc8a-ce5051575ca4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.420904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.423076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.423632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.424141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.438398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.441830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.462661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1038/2079] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:07.538288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495046e0-2cc3-41d3-a56f-a58ba9517072 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.557660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.558815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.559551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.560025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.561335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.566074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.620084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1042/2080] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:07 2022] POST /identity/v3/projects => generated 409 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:07.623343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.644648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1724c-9bb4-48d5-bbdc-9bf9ea564011 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.733689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:07.791384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.792717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.793203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.793695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.795001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.799471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.847644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1039/2081] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:07 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:07.856151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.888024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05746a66-93f2-42c3-8f04-fcd114dd969d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.905061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.907079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.908080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.909042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:07.911310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:07.915817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:07.950745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1043/2082] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:07.952439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:07.971081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f3164-5760-4459-99f5-478d52fdf3de admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:07.988690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:07.990134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:07.990637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:07.991099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] PATH_INFO: `/v3/projects/f787003ac96d46c4995dc593acdf2efc/users/56d50ab2eb7b4003b194aaf2a471e488/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:08.031852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f787003ac96d46c4995dc593acdf2efc, user_id=56d50ab2eb7b4003b194aaf2a471e488, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:08.038316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:08.067919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1040/2083] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:07 2022] PUT /identity/v3/projects/f787003ac96d46c4995dc593acdf2efc/users/56d50ab2eb7b4003b194aaf2a471e488/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:08.075145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:08.108485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4c4db-50fa-44c3-b337-326f2cb1b0e7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:08.179039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:08.269420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:08.270617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:08.271105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:08.271775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:08.273261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:08.275213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b90d00-d536-49ac-b82e-c1a7aa19d4bf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:08.292597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1044/2084] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:08.294884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:08.311207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214b82-6d1c-4ee0-8c67-fecd0e7c064d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:08.319648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:08.320652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:08.320933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:08.321209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] PATH_INFO: `/v3/projects/f787003ac96d46c4995dc593acdf2efc/users/56d50ab2eb7b4003b194aaf2a471e488/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:08.323780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f787003ac96d46c4995dc593acdf2efc, user_id=56d50ab2eb7b4003b194aaf2a471e488, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:08.327161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:08.356036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1041/2085] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:08 2022] PUT /identity/v3/projects/f787003ac96d46c4995dc593acdf2efc/users/56d50ab2eb7b4003b194aaf2a471e488/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:08.372596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:08.372887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:08.373181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:08.428169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] MFA Rules not processed for user `56d50ab2eb7b4003b194aaf2a471e488`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:08.429126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:08.628389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1045/2086] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:08.649474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:08.662291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:08.791187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b115e0b-3fd9-47d8-9701-4e642616526c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:08.798562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:08.799974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:08.800610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:08.801228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:08.830681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:08.836434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898db23-9310-45e4-8cd7-b288ff0ad1eb service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:08.872770 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1042/2087] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:09.183236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:09.200186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b59c33-e71c-4c49-b961-4751978b69f4 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:09.213104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:09.214307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:09.214807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:09.215460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:09.322378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:09.324365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:09.348562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1046/2088] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:13.580814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:13.581342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:13.581495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:13.634901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] MFA Rules not processed for user `cf7a76ad651d4f7996116c942103a496`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:13.637110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:13.719721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1043/2089] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:13.724366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:13.725786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:13.798931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:13.821177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee3437-45bb-42f9-b950-2d4fade583ab service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:13.826966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:13.827690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:13.827974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:13.828248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:13.828953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:13.830919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:13.860224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1047/2090] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:13 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:13.864019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:13.875605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1349fed-58e1-4982-998f-4fe8a06773bb None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:13.894233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:13.948054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:13.948825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:13.949104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:13.949413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:13.950135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:13.952149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:13.993296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1044/2091] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:13 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:13.997603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:14.009408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4540c65-c8ed-4eba-89fb-b3c1eb99dd2e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:14.014253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:14.014931 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:14.015347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:14.015679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:14.016428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:14.018067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:14.029688 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1048/2092] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:14.034646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:14.047568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a897b6a-ca5b-415c-8fc6-07bd9276e2aa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:14.052514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:14.053406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:14.053685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:14.053958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] PATH_INFO: `/v3/projects/2ef1937b964b497ab2734d543544d5ad/users/e8e3067ffbd74e1ea54a2be9fa570dc5/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:14.071415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef1937b964b497ab2734d543544d5ad, user_id=e8e3067ffbd74e1ea54a2be9fa570dc5, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:14.074195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:14.088415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1045/2093] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:14 2022] PUT /identity/v3/projects/2ef1937b964b497ab2734d543544d5ad/users/e8e3067ffbd74e1ea54a2be9fa570dc5/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:14.092527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:14.104355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df4905-3601-46d7-aa31-9515a62cedbb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:14.171860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:14.172500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:14.172971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:14.173108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:14.173819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:14.175377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:14.190439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1049/2094] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:14.192126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:14.202976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f2b73-1876-4857-a08f-87586cea0d89 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:14.207990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:14.208894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:14.209167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:14.209436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] PATH_INFO: `/v3/projects/2ef1937b964b497ab2734d543544d5ad/users/e8e3067ffbd74e1ea54a2be9fa570dc5/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:14.211667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef1937b964b497ab2734d543544d5ad, user_id=e8e3067ffbd74e1ea54a2be9fa570dc5, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:14.214330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9c44-7a1c-48a4-ba9d-062324704bc0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:14.225961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1046/2095] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:14 2022] PUT /identity/v3/projects/2ef1937b964b497ab2734d543544d5ad/users/e8e3067ffbd74e1ea54a2be9fa570dc5/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:16.570344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:16.575159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:16.636666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070f4f-b87c-4b9c-b5e8-ec2a48eb4e83 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:16.848850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:16.850332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:16.850332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:16.850675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] PATH_INFO: `/v3/users/f4ec145ed78f4a3990cb2d97555c1987` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:16.852830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ec145ed78f4a3990cb2d97555c1987)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:16.856426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:17.018119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ec145ed78f4a3990cb2d97555c1987'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:17.036641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ec145ed78f4a3990cb2d97555c1987'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:17.040512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ec145ed78f4a3990cb2d97555c1987'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:17.044099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:17.076078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ec145ed78f4a3990cb2d97555c1987'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:17.095304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ec145ed78f4a3990cb2d97555c1987'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:17.137899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1050/2096] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:16 2022] DELETE /identity/v3/users/f4ec145ed78f4a3990cb2d97555c1987 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:17.156146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:17.156750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:17.156946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:17.199952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:17.202584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:17.375024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1047/2097] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:17.410244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:17.431083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:17.640301 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7d463-7084-45af-a952-37e956a50c3c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:17.644204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:17.645002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:17.645282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:17.646425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:17.658589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:17.660321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:17.675532 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1051/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:19.746718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:19.748528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:19.782696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a182947-114e-48a3-acbd-38fd4313a4dc None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:19.809200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:19.811030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:19.812540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:19.812540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:19.812540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] PATH_INFO: `/v3/projects/ae3b770f0ffa4fb58a6c5d4f0d57bc0b` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:19.813414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:19.841165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3b770f0ffa4fb58a6c5d4f0d57bc0b)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:19.851943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:19.858988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792bcb-82c0-4c12-baca-5749769ed2d5 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:19.885644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:19.891281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:19.892130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:19.892979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] PATH_INFO: `/v3/users/3c867d69f3af4cb4abd035e9a058b686` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:19.902255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c867d69f3af4cb4abd035e9a058b686)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:19.906477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:19.930536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3b770f0ffa4fb58a6c5d4f0d57bc0b', 'request_id': 'req-55e66080-9803-4007-9c3b-5ad468e58ea6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:19.945637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3b770f0ffa4fb58a6c5d4f0d57bc0b', 'request_id': 'req-55e66080-9803-4007-9c3b-5ad468e58ea6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:19.958984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3b770f0ffa4fb58a6c5d4f0d57bc0b', 'request_id': 'req-55e66080-9803-4007-9c3b-5ad468e58ea6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:19.967709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:20.003916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] The token cache is being invalidate because project ae3b770f0ffa4fb58a6c5d4f0d57bc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:20.004758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.007089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-55e66080-9803-4007-9c3b-5ad468e58ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.016515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1048/2099] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:19 2022] DELETE /identity/v3/projects/ae3b770f0ffa4fb58a6c5d4f0d57bc0b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:20.018900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb63a5-dd38-4760-8720-b9e99ae9949e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:20.019349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb63a5-dd38-4760-8720-b9e99ae9949e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:20.020325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb63a5-dd38-4760-8720-b9e99ae9949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:20.113145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f5bb63a5-dd38-4760-8720-b9e99ae9949e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:20.114558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f5bb63a5-dd38-4760-8720-b9e99ae9949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:20.144636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c867d69f3af4cb4abd035e9a058b686'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.187983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c867d69f3af4cb4abd035e9a058b686'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.192794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c867d69f3af4cb4abd035e9a058b686'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.257107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c867d69f3af4cb4abd035e9a058b686'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.279662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1049/2100] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:20.286587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:20.286758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:20.287007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:20.304148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4cba49fd-f126-4b49-ab7e-c4a7e9e36f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c867d69f3af4cb4abd035e9a058b686'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:20.324397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1052/2101] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:19 2022] DELETE /identity/v3/users/3c867d69f3af4cb4abd035e9a058b686 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:20.329914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:20.331805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:20.332923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:20.337048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:20.348106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:20.437088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:20.439026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:20.555789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1050/2102] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:20.557192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:20.608084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b250-5f43-4603-ba8b-24de276fd281 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:20.644301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:20.654035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:20.657246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:20.659536 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:20.661279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:20.664489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:20.667103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:20.673007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:20.730348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1053/2103] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:20.731298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1051/2104] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:20.748573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:20.774476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:20.777465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:20.799170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc455297-3e26-4272-8aff-091770bc7cc3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:20.822128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:20.822917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:20.823243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:20.823646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:20.827522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:20.833676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:20.925378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0423d-2f2d-426d-b62c-b20d8906a42d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:20.943020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1054/2105] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:20 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:20.948200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:21.022961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:21.042480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:21.050954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.050954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.051189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.114597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4c31-8355-43a5-b7e2-c012a9103185 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:21.143997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:21.150228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.150442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.150773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.154183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:21.171557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:21.237154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:21.239038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:21.255400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1055/2106] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:20 2022] POST /identity/v3/users => generated 291 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:21.260457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:21.268076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1052/2107] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:21.285862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33614df-7715-438a-a867-2594a6ddfe62 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:21.318228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:21.319066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.320038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.320038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.320568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:21.332869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:21.370276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1056/2108] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:21.371084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:21.385732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c30339-fb46-4b23-bf27-2cf48c84d5e8 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:21.391455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:21.392517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.392907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.393341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.394199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:21.396084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893178f-78e7-48fe-b797-cab6d41c67bb admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:21.421743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:21.421945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1053/2109] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:21.455787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbf319-6a81-4a8f-ad1f-016a7d901d44 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:21.473018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:21.474396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.474721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.475115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] PATH_INFO: `/v3/projects/d8921e8d62e24ef4898fb1f142377ae6/users/89db0559dac64838991ddd1778192427/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.514550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8921e8d62e24ef4898fb1f142377ae6, user_id=89db0559dac64838991ddd1778192427, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:21.520520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:21.565507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1057/2110] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:21 2022] PUT /identity/v3/projects/d8921e8d62e24ef4898fb1f142377ae6/users/89db0559dac64838991ddd1778192427/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:21.587246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:21.587955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:21.588460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:21.669132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] MFA Rules not processed for user `89db0559dac64838991ddd1778192427`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:21.670107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:21.822457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1054/2111] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:21.828489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.006404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec8ad-8d0d-4cb5-bcae-9ff9ea461986 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.013047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.013420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.013704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.014037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.014758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.017016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.072677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1058/2112] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:21 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:22.074597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.093145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cf014-0609-44ce-bdff-d8e4f7067f7c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.231299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.234486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.238126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.238126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.238788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.242001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.276630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:22.297851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.298576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1055/2113] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:22 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:22.319603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.323197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62629ae-5e1b-4b87-81d1-c8d73b0769ce admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.338041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:22.365616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.366704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.369193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.369638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.370929 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.377288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.383669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:22.406479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fa241-5254-4c79-b978-3492e657d8a3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.416375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1059/2114] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:22.420324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.434437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.435359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.435670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.436125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] PATH_INFO: `/v3/projects/fff51e7d46e14bdfa40052adb150384e` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.438741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff51e7d46e14bdfa40052adb150384e)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.440768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.440942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729735-19fc-48d9-8300-425c94ee516e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.460409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.461124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.461399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.461671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.462526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.464210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.488884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1060/2115] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:22.493562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.516669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6d9a-8ebf-4cc7-b7f3-2ca7852221f4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.541152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff51e7d46e14bdfa40052adb150384e', 'request_id': 'req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:22.557337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff51e7d46e14bdfa40052adb150384e', 'request_id': 'req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:22.589649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff51e7d46e14bdfa40052adb150384e', 'request_id': 'req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:22.620344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] The token cache is being invalidate because project fff51e7d46e14bdfa40052adb150384e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:22.620641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:22.623049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:22.629709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1056/2116] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:22 2022] DELETE /identity/v3/projects/fff51e7d46e14bdfa40052adb150384e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:22.634429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.637298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:22.665153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.667111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.667657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.668263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] PATH_INFO: `/v3/projects/30fe59ab221e4d35ab73444674841c76/users/c91908f706854219ad3082c4de692fee/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.701833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fe59ab221e4d35ab73444674841c76, user_id=c91908f706854219ad3082c4de692fee, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.705423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd81477-cb1f-4f41-be61-bd1c045970a4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.726751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1061/2117] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:22 2022] PUT /identity/v3/projects/30fe59ab221e4d35ab73444674841c76/users/c91908f706854219ad3082c4de692fee/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:22.738753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.739178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.739625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.772604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5dc2f-f75e-4a3c-9b50-753b1fa9ad0f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:22.797513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] MFA Rules not processed for user `c91908f706854219ad3082c4de692fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:22.799069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:22.920032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:22.923470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:22.924292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:22.924986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] PATH_INFO: `/v3/users/56d50ab2eb7b4003b194aaf2a471e488` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:22.928890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d50ab2eb7b4003b194aaf2a471e488)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:22.932870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:22.943177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1062/2118] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:22.947818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:22.950410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:22.999341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:23.008352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ade9e-5633-4645-9a07-793f63c3ff0b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.024765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.025400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.025671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.025945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.029715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.036316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.042740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d50ab2eb7b4003b194aaf2a471e488'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:23.061286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d50ab2eb7b4003b194aaf2a471e488'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:23.097035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d50ab2eb7b4003b194aaf2a471e488'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:23.098895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d50ab2eb7b4003b194aaf2a471e488'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:23.111298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1063/2119] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:22 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:23.115724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.116244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f48069-63e7-418d-bc08-d5717b15175c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d50ab2eb7b4003b194aaf2a471e488'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:23.120766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:23.133216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1057/2120] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:22 2022] DELETE /identity/v3/users/56d50ab2eb7b4003b194aaf2a471e488 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:23.256198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:23.293891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59eaf9-ea2f-4b2f-b489-8644ed1e728d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.399369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.400290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.400795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.401203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.402091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.416113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.416447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.416732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.416949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.451238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1064/2121] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:23 2022] POST /identity/v3/users => generated 299 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:23.454491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.497207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2861-0624-4ebc-9aa5-761cea2edbd6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.508589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] MFA Rules not processed for user `e8e3067ffbd74e1ea54a2be9fa570dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:23.510271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:23.514917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.515740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.516025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.516300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.517131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.518678 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.552537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1065/2122] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:23.553452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.585964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac0b7-4b9c-4687-a11a-3b33dee7c17a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.604703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.605663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.605942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.606217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] PATH_INFO: `/v3/projects/35fa74ebad4f4d4ba97398e6a614d965/users/14024939849949578b9b55b29939b11c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.646527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fa74ebad4f4d4ba97398e6a614d965, user_id=14024939849949578b9b55b29939b11c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.651655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.675983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1066/2123] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:23 2022] PUT /identity/v3/projects/35fa74ebad4f4d4ba97398e6a614d965/users/14024939849949578b9b55b29939b11c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:23.683588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.752319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1058/2124] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:23.758156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.760778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:23.829621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a00f7-970b-4146-ad22-56422eed5f6f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.834743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.835440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.835740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.836342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.837205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.840163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.864320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1067/2125] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:23.868823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.894510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc485dc-2b6e-4f1a-9d20-e2afa79d1cc1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.900431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:23.901632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:23.901979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:23.902378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] PATH_INFO: `/v3/projects/35fa74ebad4f4d4ba97398e6a614d965/users/14024939849949578b9b55b29939b11c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:23.903420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eebad-5307-4859-b233-d6c319bce322 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:23.908924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fa74ebad4f4d4ba97398e6a614d965, user_id=14024939849949578b9b55b29939b11c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:23.911999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:23.935336 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1068/2126] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:23 2022] PUT /identity/v3/projects/35fa74ebad4f4d4ba97398e6a614d965/users/14024939849949578b9b55b29939b11c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:23.936352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:23.948820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:23.958205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:24.007696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:24.022843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:24.024219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.024510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.024792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.026920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.097140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28face08-51eb-4bfd-addf-2c6dcf4221e8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:24.111275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:24.113605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.114113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.114831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] PATH_INFO: `/v3/projects/f787003ac96d46c4995dc593acdf2efc` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.118887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f787003ac96d46c4995dc593acdf2efc)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:24.122992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:24.132903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:24.134872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9995124-306d-45c2-8cf6-65552fe25b85 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:24.148105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1059/2127] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:24.149648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.149937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.150327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.191956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] MFA Rules not processed for user `14024939849949578b9b55b29939b11c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:24.193407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.264684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f787003ac96d46c4995dc593acdf2efc', 'request_id': 'req-c097a4e6-98db-4356-9a49-63620203fb22'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:24.280215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f787003ac96d46c4995dc593acdf2efc', 'request_id': 'req-c097a4e6-98db-4356-9a49-63620203fb22'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:24.289651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f787003ac96d46c4995dc593acdf2efc', 'request_id': 'req-c097a4e6-98db-4356-9a49-63620203fb22'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:24.327622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] The token cache is being invalidate because project f787003ac96d46c4995dc593acdf2efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:24.327622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:24.328476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c097a4e6-98db-4356-9a49-63620203fb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:24.330548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1069/2128] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:23 2022] DELETE /identity/v3/projects/f787003ac96d46c4995dc593acdf2efc => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:24.337446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e84e-865e-45ab-a5df-71c72d974f01 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.337792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e84e-865e-45ab-a5df-71c72d974f01 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.338093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053e84e-865e-45ab-a5df-71c72d974f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.380070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b053e84e-865e-45ab-a5df-71c72d974f01 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:24.382512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b053e84e-865e-45ab-a5df-71c72d974f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.421896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1060/2129] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:24.439123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:24.444190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.537008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1070/2130] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:24.548351 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.548624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.548905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.571640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:24.572974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.582677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1071/2131] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:24.587826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:24.588419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23193416-ed0a-4245-bc48-79f100e4ddca None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:24.602148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:24.633020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bb2da-8946-4a30-97ec-595def4671ae None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:24.642036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:24.643165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.643641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.644040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.645656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:24.645656 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:24.650219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:24.690047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1072/2132] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:17:24.691455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:24.780561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aa26e-8530-4372-9336-fc5ae646ff17 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:24.800078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:24.800783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.801066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.801436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.802060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:24.806028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:24.810785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:24.811936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:24.812333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:24.812884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:24.821514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:24.900627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1073/2133] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:24 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:24.906992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:24.947945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:25.027135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:25.029736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:25.074880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1061/2134] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:25.116148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72dc06-4e3d-4e63-880b-01759cdbf9d3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:25.125164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:25.126225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:25.126519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:25.126795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:25.127797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:25.131063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:25.180437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1074/2135] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:24 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:25.184855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:25.216865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c2277-60c8-4886-ae57-30281d9a468a service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:25.253808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:25.257156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:25.258475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:25.258780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:25.264559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:25.270196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:25.324834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1062/2136] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:25.331734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:25.352376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fded0f-8793-41e6-a3f6-3d2f759be023 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:25.384153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:25.384875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:25.385242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:25.386207 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:25.392114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:25.394074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ecb64-53d7-4bb9-ac1d-4aee06f4d604 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:25.427791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1075/2137] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:25.434602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:25.451335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2bf45-b0f4-4dd3-a8cb-4370900803aa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:25.466937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:25.468977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:25.468977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:25.468977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] PATH_INFO: `/v3/projects/5b310f95c6f74cd98d860aadb1e948fc/users/5e192ccbdf4d45a78352fe61c80c15b3/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:25.512078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b310f95c6f74cd98d860aadb1e948fc, user_id=5e192ccbdf4d45a78352fe61c80c15b3, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:25.515580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:25.564439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1063/2138] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:25 2022] PUT /identity/v3/projects/5b310f95c6f74cd98d860aadb1e948fc/users/5e192ccbdf4d45a78352fe61c80c15b3/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:25.576394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:25.576869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:25.577498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:25.640844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] MFA Rules not processed for user `5e192ccbdf4d45a78352fe61c80c15b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:25.642571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:25.866475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1076/2139] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:25.868014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.053917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a1919-e3af-4cb0-8745-1a301320ed89 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.072476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.073211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.073491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.073785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.074608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.076754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.147263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1064/2140] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:25 2022] POST /identity/v3/projects => generated 381 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:26.150095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.192135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d5ac6-ff03-413a-9808-96801e66ef34 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.192282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.192282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:26.228593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:26.264468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.265429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.266422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.266898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.268087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.271020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.315056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1077/2141] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:26 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:26.317574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.331418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889550bf-af8f-4b66-8bac-97d2d2c4f85d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.338798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.339723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.340024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.340306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.341071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.342916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.346202 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:26.362499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1078/2142] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:26.366873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.390478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174794-c206-4a93-a7c9-116afa1db67e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.399813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889dea-e72e-4fb6-9008-842d27cbad43 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.403571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.403923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.404489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.404489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] PATH_INFO: `/v3/projects/ad6608653fec40bcb2525858c868ecfb/users/9d7b8a794bc44f958f714716a716eb06/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.420318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.421818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.422310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.422791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] PATH_INFO: `/v3/users/caa581494f95446ba423c136c378d2e3` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.424409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa581494f95446ba423c136c378d2e3)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.426656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.446758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6608653fec40bcb2525858c868ecfb, user_id=9d7b8a794bc44f958f714716a716eb06, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.458001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.495164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1079/2143] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:26 2022] PUT /identity/v3/projects/ad6608653fec40bcb2525858c868ecfb/users/9d7b8a794bc44f958f714716a716eb06/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:26.501313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.548637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa581494f95446ba423c136c378d2e3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:26.565834 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa581494f95446ba423c136c378d2e3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:26.600450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa581494f95446ba423c136c378d2e3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:26.601377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa581494f95446ba423c136c378d2e3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:26.618811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b27d18eb-39c0-4b2f-b32e-bd3053531ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa581494f95446ba423c136c378d2e3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:26.631134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1065/2144] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:26 2022] DELETE /identity/v3/users/caa581494f95446ba423c136c378d2e3 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:26.636184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.636184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.636525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.675826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:26.679947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:26.708786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237188b-f0a6-414b-93c9-5c5ace584938 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:26.832672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1066/2145] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:26.892078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.894134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:26.910320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:26.913628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.914848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.915273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.917224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:26.922907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad4386-3016-4ac0-b39f-58e6de1430d2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:26.960671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1080/2146] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:26.962470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:26.968079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:26.968243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:26.976550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1081/2147] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:17:26 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:17:26.976550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:26.987450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.017814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:27.043827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38ec0f-eacc-4e21-8e32-47619c6d85d7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.055787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.057915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.059361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.059741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] PATH_INFO: `/v3/projects/ad6608653fec40bcb2525858c868ecfb/users/9d7b8a794bc44f958f714716a716eb06/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.101277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6608653fec40bcb2525858c868ecfb, user_id=9d7b8a794bc44f958f714716a716eb06, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:27.104243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02d94e-76eb-4219-a807-73b81082a43a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:27.106891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60096942-c446-4ffd-b35d-4354e26719e9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.114318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.115148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.115335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.115650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.134505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1082/2148] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:26 2022] PUT /identity/v3/projects/ad6608653fec40bcb2525858c868ecfb/users/9d7b8a794bc44f958f714716a716eb06/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:27.137708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32138e3f-4343-4154-960f-820fe24e033d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.138000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32138e3f-4343-4154-960f-820fe24e033d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.138539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32138e3f-4343-4154-960f-820fe24e033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.160461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:27.164313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55ba47-114e-42fc-9337-f968bae9e319 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:27.189798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1067/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:27.193019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.194696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.195159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.208859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-32138e3f-4343-4154-960f-820fe24e033d None None] MFA Rules not processed for user `76fd650362ab4b4f959c08d46e1827ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:27.217376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-32138e3f-4343-4154-960f-820fe24e033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.265301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] MFA Rules not processed for user `9d7b8a794bc44f958f714716a716eb06`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:27.269232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.365619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:27.397804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1083/2150] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:17:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:27.400502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32138e3f-4343-4154-960f-820fe24e033d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:27.406836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-32138e3f-4343-4154-960f-820fe24e033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.503556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1068/2151] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:27.503556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:27.505093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.537744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d22a11-6fc4-41cc-8a57-5c81db0b200b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.549347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.550828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.551237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.551655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.563841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:27.566936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:27.600696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1069/2152] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:27.604004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:27.606331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.609498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32138e3f-4343-4154-960f-820fe24e033d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.623448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.624660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.625010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.625361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] PATH_INFO: `/v3/users/c3077763acd8487291f7005bd1da77c2` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.627037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3077763acd8487291f7005bd1da77c2)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:27.635009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:27.779158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3077763acd8487291f7005bd1da77c2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:27.805388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3077763acd8487291f7005bd1da77c2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:27.806506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3077763acd8487291f7005bd1da77c2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:27.818094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4df9b4-ccc5-4919-bec3-82dbcbbeb62f service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.833378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.834240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.834625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.834999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.837955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.855711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3077763acd8487291f7005bd1da77c2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:27.876287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3077763acd8487291f7005bd1da77c2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:27.889973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1084/2153] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:27 2022] DELETE /identity/v3/users/c3077763acd8487291f7005bd1da77c2 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:27.889973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:27.889973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:27.980832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c706a8-6434-4728-86fd-afb0575e91fe admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:27.988247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:27.989133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:27.990139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:27.990526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:27.994359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.010038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:28.011616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2b47d-d0cf-42fb-a210-31210d62b7ae service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:28.024953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1070/2154] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:28.026430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.026724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.027004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.046978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:28.072802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:28.073597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] MFA Rules not processed for user `cf7a76ad651d4f7996116c942103a496`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:28.078573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.153381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:28.156410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:28.190637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1085/2155] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:28.224364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1071/2156] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:28.300864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:28.303836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.342677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:28.345337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.484510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a18269-3386-4a47-8c62-3ac9cd9bdb8a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:28.489821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:28.490738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.491042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.491321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] PATH_INFO: `/v3/projects/41b8b45e25af4c498afb80b94362fe95` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.492624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b8b45e25af4c498afb80b94362fe95)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:28.494004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318708b-6d85-480e-b402-b928fe0000cf service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:28.494813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:28.515360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:28.516520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.516823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.517137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] PATH_INFO: `/v3/projects/0cae41d6b94a4850b9e699be985a295f` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.521967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cae41d6b94a4850b9e699be985a295f)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:28.527642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:28.591296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b8b45e25af4c498afb80b94362fe95', 'request_id': 'req-edbc5672-974d-4e70-8ea6-8dbb8fce0530'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.603395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b8b45e25af4c498afb80b94362fe95', 'request_id': 'req-edbc5672-974d-4e70-8ea6-8dbb8fce0530'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.611813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b8b45e25af4c498afb80b94362fe95', 'request_id': 'req-edbc5672-974d-4e70-8ea6-8dbb8fce0530'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.616404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cae41d6b94a4850b9e699be985a295f', 'request_id': 'req-8cbef754-c4e4-449e-ae1d-83bdba693f06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.625854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] The token cache is being invalidate because project 41b8b45e25af4c498afb80b94362fe95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:28.626158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.626905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-edbc5672-974d-4e70-8ea6-8dbb8fce0530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.629901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1072/2157] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:28 2022] DELETE /identity/v3/projects/41b8b45e25af4c498afb80b94362fe95 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:28.635119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.635409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.635720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.636288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cae41d6b94a4850b9e699be985a295f', 'request_id': 'req-8cbef754-c4e4-449e-ae1d-83bdba693f06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.649891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cae41d6b94a4850b9e699be985a295f', 'request_id': 'req-8cbef754-c4e4-449e-ae1d-83bdba693f06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.677387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:28.679155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.681271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] The token cache is being invalidate because project 0cae41d6b94a4850b9e699be985a295f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:28.681890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.683059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8cbef754-c4e4-449e-ae1d-83bdba693f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:28.686748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1086/2158] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:28 2022] DELETE /identity/v3/projects/0cae41d6b94a4850b9e699be985a295f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:28.748373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:28.811189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1073/2159] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:28.811882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.812594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.813111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.847547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:28.850527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:28.892837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1087/2160] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:28.897910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:28.914703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3659d3c-0615-4245-825b-995c89fec88f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:28.930584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:28.931555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.931803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.932361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.933188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:28.933671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:28.935534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:28.960986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1074/2161] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:17:28.966877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:28.979791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e943c-a7c7-4a01-b828-e78fea1ce70a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:28.995949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:28.997070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:28.997366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:28.997651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:28.998902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.001611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.046298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1088/2162] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:28 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:29.050903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.068156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaeaafd-a2a8-4120-9008-a5379c70b72b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.207273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.210918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.248200 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.249486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.249968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.250462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.251959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.256890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.288275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:29.337407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1075/2163] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:29 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:29.340268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.343149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e065f-c73b-445e-ae57-96416835c29a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.347106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.347927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.348268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.348637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.350978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.376022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca32c1-fab7-471c-b4af-e7ad4a65a4e1 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.393757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.398417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.398417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.398417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.398417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.398417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.420554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1076/2164] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:29.425155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.447766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb800779-e2f4-46dc-8608-2f133954ad78 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.455955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.456944 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.457231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.457516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.458570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.462087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.486131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1077/2165] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:29.491429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.506605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a23d3b9-4ede-4af2-868c-3eb05a0777de admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.514037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.517231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a499c-4ddc-4a5a-996b-e5cb1c169bfb service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.521844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.522893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.523179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.523445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] PATH_INFO: `/v3/projects/e6f6c1e0d8734b1f975dcf1a5b57690e/users/c1c1afef4cb0403cbb56c34942f3b2a7/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.546836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1089/2166] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:29.582145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f6c1e0d8734b1f975dcf1a5b57690e, user_id=c1c1afef4cb0403cbb56c34942f3b2a7, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.587582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.602859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1078/2167] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:29 2022] PUT /identity/v3/projects/e6f6c1e0d8734b1f975dcf1a5b57690e/users/c1c1afef4cb0403cbb56c34942f3b2a7/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:29.612189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.612514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.613596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.641864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.644290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.683147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] MFA Rules not processed for user `c1c1afef4cb0403cbb56c34942f3b2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:29.684681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.755937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ecfa8-c20a-4ae7-ade6-afaf84234bf9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.763277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:29.764352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.764831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.765354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] PATH_INFO: `/v3/users/5e192ccbdf4d45a78352fe61c80c15b3` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.767982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e192ccbdf4d45a78352fe61c80c15b3)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:29.771515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:29.790370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:29.796551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1090/2168] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:29.818763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:29.820539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.863374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e192ccbdf4d45a78352fe61c80c15b3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:29.877819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e192ccbdf4d45a78352fe61c80c15b3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:29.899281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e192ccbdf4d45a78352fe61c80c15b3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:29.900530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e192ccbdf4d45a78352fe61c80c15b3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:29.914830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca37cbeb-8cd6-40f6-addf-8abfb289d6b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e192ccbdf4d45a78352fe61c80c15b3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:29.929736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1079/2169] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:29 2022] DELETE /identity/v3/users/5e192ccbdf4d45a78352fe61c80c15b3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:29.933306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:29.935393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:29.935860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:29.964814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:29.966169 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:29.986535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffeac0-3c4c-4ba5-8d1f-f1f9f74a102e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:29.998417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:30.076581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1080/2170] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:30.091049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:30.093277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:30.136138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:30.136958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:30.137472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:30.137768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:30.140064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:30.254343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3efea8-f088-4c09-86c3-28fba9daea33 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:30.265419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:30.266625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:30.267996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:30.268662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:30.316846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:30.326795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:30.329853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:30.333383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:30.342106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1081/2171] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:30.355327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1091/2172] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:30.888998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:30.891634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:30.925356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea48111-d01e-45c0-8ff1-4f27b3f001e7 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:30.936978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:30.938931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:30.939277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:30.939806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] PATH_INFO: `/v3/projects/5b310f95c6f74cd98d860aadb1e948fc` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:30.942766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b310f95c6f74cd98d860aadb1e948fc)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:30.948593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:31.006088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:31.035045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b310f95c6f74cd98d860aadb1e948fc', 'request_id': 'req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.045023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b310f95c6f74cd98d860aadb1e948fc', 'request_id': 'req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.053234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b310f95c6f74cd98d860aadb1e948fc', 'request_id': 'req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.067836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] The token cache is being invalidate because project 5b310f95c6f74cd98d860aadb1e948fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:31.068137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.068892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.070518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1082/2173] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:30 2022] DELETE /identity/v3/projects/5b310f95c6f74cd98d860aadb1e948fc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:31.075772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:31.096214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b348c2b-5e46-4c66-870d-d039d5b94a32 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:31.204265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:31.210105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:31.210431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:31.210666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] PATH_INFO: `/v3/users/9d7b8a794bc44f958f714716a716eb06` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:31.212212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7b8a794bc44f958f714716a716eb06)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:31.214312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:31.295324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7b8a794bc44f958f714716a716eb06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.296976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:31.309127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7b8a794bc44f958f714716a716eb06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.310501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7b8a794bc44f958f714716a716eb06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.332698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7b8a794bc44f958f714716a716eb06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.344672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-13a1183d-a538-4417-b2c3-0e2bc67ba210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7b8a794bc44f958f714716a716eb06'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.353029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1092/2174] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:31 2022] DELETE /identity/v3/users/9d7b8a794bc44f958f714716a716eb06 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:31.721559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:31.723521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:31.744137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596abb1a-09d0-46a3-8dbd-ceec7021c6c6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:31.820405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:31.821308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:31.821677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:31.821909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] PATH_INFO: `/v3/projects/ad6608653fec40bcb2525858c868ecfb` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:31.823616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6608653fec40bcb2525858c868ecfb)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:31.826197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:31.885984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6608653fec40bcb2525858c868ecfb', 'request_id': 'req-c96064d6-7c5f-484f-9591-ecf7ffd6de61'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.894795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6608653fec40bcb2525858c868ecfb', 'request_id': 'req-c96064d6-7c5f-484f-9591-ecf7ffd6de61'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.902926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6608653fec40bcb2525858c868ecfb', 'request_id': 'req-c96064d6-7c5f-484f-9591-ecf7ffd6de61'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.915447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] The token cache is being invalidate because project ad6608653fec40bcb2525858c868ecfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:31.915816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.916449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c96064d6-7c5f-484f-9591-ecf7ffd6de61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:31.918309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1083/2175] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:31 2022] DELETE /identity/v3/projects/ad6608653fec40bcb2525858c868ecfb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:31.986350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:31.986977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:31.986977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.005383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:32.006637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:32.077449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1093/2176] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:32.085248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.085539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.085813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.108213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:32.109541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:32.118577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1084/2177] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:32.123164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.133619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ae6e7-f839-4b7b-bf96-1e04f4c6cee7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.138833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.139454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.139770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.140044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.140787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:32.141094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.142762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.170687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1094/2178] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:32.175322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.185262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c9e7a-4e3f-40c6-99dc-332419d90a41 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.191913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.193068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.193568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.194076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.195421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.199306 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.248366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1085/2179] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:32.249621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.260867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc18e0-b481-42c8-98c6-2ba30e3a6fa3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.342233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.343289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.343651 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.344069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.344775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.347154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.371377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1095/2180] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:32.376068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.387786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673309-032a-4875-aaff-386c79e0ed26 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.394522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.395261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.395556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.395840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.396587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.398261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.415571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1086/2181] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:32.415940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.428003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9ef00-7fc7-4451-b3ba-556136dc943d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.432914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.433696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.433984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.434263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.435009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.436757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57708b-013b-475d-815b-3b92ad2ec467 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.454057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1096/2182] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:32.459246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.466473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:32.478889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a73722-142d-45e4-9161-03f159248518 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.487808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.489588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.491531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.491531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] PATH_INFO: `/v3/projects/dbd79b1044514826a260ce190a2b50d1/users/5bfbc0ea10f84e0bba4e916cf5a01061/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.514596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd79b1044514826a260ce190a2b50d1, user_id=5bfbc0ea10f84e0bba4e916cf5a01061, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.519971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.537284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1087/2183] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:32 2022] PUT /identity/v3/projects/dbd79b1044514826a260ce190a2b50d1/users/5bfbc0ea10f84e0bba4e916cf5a01061/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:32.553976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.554340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.554629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.586586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] MFA Rules not processed for user `5bfbc0ea10f84e0bba4e916cf5a01061`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:32.587947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:32.617084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:32.680177 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1097/2184] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:32.685445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.793965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693e20c-6c2f-4254-b178-7c6adb0d8113 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.799102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.799830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.800116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.800401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.801151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.803153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.837974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1088/2185] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:32.843376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.856832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ed87c-816d-49e6-a3bc-5ea8ee8799ac None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.935767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.936477 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.936756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.937032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.937751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.939795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:32.964295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1098/2186] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:32 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:32.969127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:32.981463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813d016-0d90-468b-a33e-21b97cb93081 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:32.987297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:32.988030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:32.988315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:32.988592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:32.989343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:32.991060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.005620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1089/2187] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:33.011938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.024658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e55bc3-4e93-4224-ae77-5121ac8c8102 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.032071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.033043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.033324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.033626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] PATH_INFO: `/v3/projects/6a3e65c6c523444b9561fc94376480d3/users/aeaad4a2166d436c8f162654285dbbea/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.052359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3e65c6c523444b9561fc94376480d3, user_id=aeaad4a2166d436c8f162654285dbbea, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.055736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.082389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1099/2188] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:33 2022] PUT /identity/v3/projects/6a3e65c6c523444b9561fc94376480d3/users/aeaad4a2166d436c8f162654285dbbea/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:33.087001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.100936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7af9d-64e5-4056-b094-7a11a1d8608b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.183296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.184096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.184384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.184661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.185463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.187159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d703b-4ed1-4f96-a413-75122f91e9eb admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.200081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1090/2189] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:33.205078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.216673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2964f-ee8b-460d-abb6-2cb6fe1dc4bc admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.222087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.223065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.223353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.223837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] PATH_INFO: `/v3/projects/6a3e65c6c523444b9561fc94376480d3/users/aeaad4a2166d436c8f162654285dbbea/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.226080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3e65c6c523444b9561fc94376480d3, user_id=aeaad4a2166d436c8f162654285dbbea, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.229185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.242229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1100/2190] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:33 2022] PUT /identity/v3/projects/6a3e65c6c523444b9561fc94376480d3/users/aeaad4a2166d436c8f162654285dbbea/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:33.254900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.255338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.255698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.293001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] MFA Rules not processed for user `aeaad4a2166d436c8f162654285dbbea`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:33.294330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.344044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:33.386320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.391111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.394304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1091/2191] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:33.411517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.416819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.478600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9fd68-abfb-4a76-b495-2eff34e30b8d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.484348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.485304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.485595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.485977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] PATH_INFO: `/v3/users/c1c1afef4cb0403cbb56c34942f3b2a7` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.487378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c1afef4cb0403cbb56c34942f3b2a7)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.489634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.512550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421112f8-8706-4dc9-8b6f-236747374b87 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.518044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:33.519154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.520284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.520778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.521236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.574906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c1afef4cb0403cbb56c34942f3b2a7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:33.584807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c1afef4cb0403cbb56c34942f3b2a7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:33.586036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c1afef4cb0403cbb56c34942f3b2a7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:33.604740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c1afef4cb0403cbb56c34942f3b2a7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:33.607097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.608739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a7d90-5aa8-4778-b11c-690a2b1d3cbc service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.618570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c1afef4cb0403cbb56c34942f3b2a7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:33.621367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1092/2192] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:33.628169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1101/2193] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:33 2022] DELETE /identity/v3/users/c1c1afef4cb0403cbb56c34942f3b2a7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:33.632954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.633246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.633525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.653043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:33.654199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.731514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1093/2194] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:33.742004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.743462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.847994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e53dd0-641c-48b4-b87b-699e6c9ccb77 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.853491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.854760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.855262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.855623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.871287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:33.873496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:33.879073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.881891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0fe0-97fc-4758-90ae-36628379d7c0 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.891078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488e02e-1fff-4f58-a636-adb070d6597a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:33.896844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1102/2195] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:33.899922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:33.901403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:33.901896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:33.902451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] PATH_INFO: `/v3/users/89db0559dac64838991ddd1778192427` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:33.904787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89db0559dac64838991ddd1778192427)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:33.908546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:33.985871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:34.033117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89db0559dac64838991ddd1778192427'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.046221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89db0559dac64838991ddd1778192427'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.104025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89db0559dac64838991ddd1778192427'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.109708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89db0559dac64838991ddd1778192427'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.122672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89db0559dac64838991ddd1778192427'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.130449 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1094/2196] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:33 2022] DELETE /identity/v3/users/89db0559dac64838991ddd1778192427 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:34.139137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.139851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.141356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.201785 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:34.204396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.304415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1103/2197] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:34.317392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:34.319679 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.395150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8daa5c-d45d-403c-8a2f-878bca07aac9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:34.399537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:34.400274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.400569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.400839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.411562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:34.413144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6fbef-4b63-4bff-b792-f5e917fdd673 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:34.424547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1095/2198] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:34.510910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:34.513243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.536059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12413f-c29a-4f50-b10c-315ea0870dac None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:34.543123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:34.544310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.547326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.547326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] PATH_INFO: `/v3/projects/e6f6c1e0d8734b1f975dcf1a5b57690e` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.547901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f6c1e0d8734b1f975dcf1a5b57690e)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:34.550059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:34.606901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:34.628941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f6c1e0d8734b1f975dcf1a5b57690e', 'request_id': 'req-9815750f-2749-4f9e-9cd3-c59d10990c0c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.643118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f6c1e0d8734b1f975dcf1a5b57690e', 'request_id': 'req-9815750f-2749-4f9e-9cd3-c59d10990c0c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.658966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f6c1e0d8734b1f975dcf1a5b57690e', 'request_id': 'req-9815750f-2749-4f9e-9cd3-c59d10990c0c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.680501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] The token cache is being invalidate because project e6f6c1e0d8734b1f975dcf1a5b57690e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:34.680825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.681918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9815750f-2749-4f9e-9cd3-c59d10990c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:34.685561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1104/2199] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:34 2022] DELETE /identity/v3/projects/e6f6c1e0d8734b1f975dcf1a5b57690e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:34.693238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.693636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.693989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.714302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:34.715687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.801585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1096/2200] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:34.811956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.812427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.812902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.844555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:34.845902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.854825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1105/2201] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:34.860774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:34.876411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a874dc3-0ad2-42cf-9395-97dce4fc969b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:34.878244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:34.882438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:34.884201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:34.886393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.887657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.888432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.890316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:34.891283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:34.896071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:34.922643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ee53a-0176-4bc7-b6db-d23d92a1adfe None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:34.927691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1097/2202] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:34.932737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:34.945753 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:34.946153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934ae27-ba16-4337-bff1-56b69046f132 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:34.947472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.947472 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.948544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] PATH_INFO: `/v3/projects/d8921e8d62e24ef4898fb1f142377ae6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.951735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:34.952538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:34.952799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:34.953035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:34.953453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8921e8d62e24ef4898fb1f142377ae6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:34.953806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:34.955830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:34.956519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.019576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1098/2203] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:34 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:35.021013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.037185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8921e8d62e24ef4898fb1f142377ae6', 'request_id': 'req-9017281c-330f-48a6-b359-8b5480226e42'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.041333 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:35.047179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8921e8d62e24ef4898fb1f142377ae6', 'request_id': 'req-9017281c-330f-48a6-b359-8b5480226e42'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.054362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8921e8d62e24ef4898fb1f142377ae6', 'request_id': 'req-9017281c-330f-48a6-b359-8b5480226e42'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.076333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] The token cache is being invalidate because project d8921e8d62e24ef4898fb1f142377ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:35.076854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.078032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.080941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1106/2204] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:34 2022] DELETE /identity/v3/projects/d8921e8d62e24ef4898fb1f142377ae6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:35.085766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.087819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:35.117146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad0fcc-3de6-4f48-9415-0c1e025ed37d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.201509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017281c-330f-48a6-b359-8b5480226e42 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.209201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.209917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.210203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.210482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.210684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.211202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.211754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.212045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.212321 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] PATH_INFO: `/v3/users/c91908f706854219ad3082c4de692fee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.213270 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.214011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91908f706854219ad3082c4de692fee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.220692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.239547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1099/2205] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:35 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:35.244710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.247769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:35.278516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109421fa-6b4d-40a2-8469-68c5d4b3d767 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.284825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.285367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:35.285790 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.286123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.286438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.287249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.289579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.300288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91908f706854219ad3082c4de692fee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.306255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1100/2206] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:35.306837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.311246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91908f706854219ad3082c4de692fee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.312057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91908f706854219ad3082c4de692fee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.318389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349f251-6369-4034-8823-1a73e1fe589b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.324653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.325347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.325626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.326080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.327302 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.328413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91908f706854219ad3082c4de692fee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.329900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.344924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3b9c153a-347f-4cd6-bee0-c40791b31e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91908f706854219ad3082c4de692fee'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:35.347123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1101/2207] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:35.351775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.353476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:35.356349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1107/2208] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:35 2022] DELETE /identity/v3/users/c91908f706854219ad3082c4de692fee => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:35.367501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50655b-bfa9-4001-8c96-f7f7f8688047 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.531862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.533183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.535394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.536305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] PATH_INFO: `/v3/projects/af6ed150de8548149f7726755c45b5dd/users/d80433d6983b49ffbd3436a1e8d9778a/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.560977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6ed150de8548149f7726755c45b5dd, user_id=d80433d6983b49ffbd3436a1e8d9778a, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.564343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.583257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1102/2209] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:35 2022] PUT /identity/v3/projects/af6ed150de8548149f7726755c45b5dd/users/d80433d6983b49ffbd3436a1e8d9778a/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:35.604188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.604710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.605306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.661892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] MFA Rules not processed for user `d80433d6983b49ffbd3436a1e8d9778a`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:35.663316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:35.813035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.816226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1108/2210] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:35.840613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:35.967067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce5f96-8f78-4e39-a26a-522fbbb739da admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:35.973735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:35.974542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:35.974904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:35.975424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:35.976466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:35.979552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:35.986620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:35.994333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.048975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ba4ba-24b3-425a-9545-8f583f22311c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.052706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.053746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.054143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.054662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] PATH_INFO: `/v3/projects/30fe59ab221e4d35ab73444674841c76` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.060709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fe59ab221e4d35ab73444674841c76)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.060985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1103/2211] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:35 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:36.061096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.061755 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.142289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fe59ab221e4d35ab73444674841c76', 'request_id': 'req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.152314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fe59ab221e4d35ab73444674841c76', 'request_id': 'req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.165571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fe59ab221e4d35ab73444674841c76', 'request_id': 'req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.186486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107676a9-8678-487b-9336-d307544d7044 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.192298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] The token cache is being invalidate because project 30fe59ab221e4d35ab73444674841c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:36.192298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.193583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.196350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1109/2212] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:35 2022] DELETE /identity/v3/projects/30fe59ab221e4d35ab73444674841c76 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:36.201868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.204767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.249798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:36.288535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.289235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.289780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.290491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.291166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.293995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.307648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace8c09-9ec4-454c-ad9b-e3d7065a5f38 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.319548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1104/2213] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:36 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:36.328330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.329553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.330681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.331019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.331272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.331455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] PATH_INFO: `/v3/users/14024939849949578b9b55b29939b11c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.332957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14024939849949578b9b55b29939b11c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.335100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.353198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e7eb7-9b22-4842-9c15-1bbd5ccdbef6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.360093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255013a-c651-433f-b336-3e431094142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.360915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255013a-c651-433f-b336-3e431094142f admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.362265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255013a-c651-433f-b336-3e431094142f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.362552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255013a-c651-433f-b336-3e431094142f admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.363344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255013a-c651-433f-b336-3e431094142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.365128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255013a-c651-433f-b336-3e431094142f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.383927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1105/2214] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:36.385485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255013a-c651-433f-b336-3e431094142f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.399823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255013a-c651-433f-b336-3e431094142f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.415702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.416328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.416614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.416897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.417720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.419428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.423646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14024939849949578b9b55b29939b11c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.433730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1106/2215] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:36.440059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14024939849949578b9b55b29939b11c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.441635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14024939849949578b9b55b29939b11c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.445350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.461504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0de78-5af0-4739-9b5b-aa54d6922427 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.468708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.471923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.472348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.473582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] PATH_INFO: `/v3/projects/0d67f4bc4dcd43d5b0f54135c19686df/users/ed73bc33ca1540d7a7e6a47135e7f01c/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.475672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14024939849949578b9b55b29939b11c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.477296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:36.496326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d67f4bc4dcd43d5b0f54135c19686df, user_id=ed73bc33ca1540d7a7e6a47135e7f01c, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.499434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.500580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-87f79ac0-17e1-4c5f-bde2-e629eb1f6aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14024939849949578b9b55b29939b11c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:36.517137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1107/2217] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:36 2022] PUT /identity/v3/projects/0d67f4bc4dcd43d5b0f54135c19686df/users/ed73bc33ca1540d7a7e6a47135e7f01c/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:36.524239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1110/2217] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:36 2022] DELETE /identity/v3/users/14024939849949578b9b55b29939b11c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:36.538814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.544119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.545101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.545393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.545676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.585523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] MFA Rules not processed for user `ed73bc33ca1540d7a7e6a47135e7f01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:36.587037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.722953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1111/2218] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:17:36.724855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.728455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1372-99a2-4375-9115-07ecf125818f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.729125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:36.831727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:36.908066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae948da-ebe7-4439-9246-397958e55f86 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:36.912871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.913782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.914061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.914336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] PATH_INFO: `/v3/users/5bfbc0ea10f84e0bba4e916cf5a01061` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.915212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:36.915691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfbc0ea10f84e0bba4e916cf5a01061)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.916813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:36.917172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:36.917554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:36.917742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.918427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:36.920782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:36.960210 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1112/2219] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:36 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:36.964950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:36.998280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfbc0ea10f84e0bba4e916cf5a01061'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.011879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfbc0ea10f84e0bba4e916cf5a01061'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.028642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfbc0ea10f84e0bba4e916cf5a01061'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.030013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfbc0ea10f84e0bba4e916cf5a01061'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.040682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfbc0ea10f84e0bba4e916cf5a01061'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.065049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1108/2220] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:36 2022] DELETE /identity/v3/users/5bfbc0ea10f84e0bba4e916cf5a01061 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:37.065049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.065049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.093549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:37.137659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f090e-701a-4424-a142-ef3f0faae812 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.207123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed43588-a55e-4253-beba-3744d7689eb3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.216639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.218283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.218946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.219936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] PATH_INFO: `/v3/projects/35fa74ebad4f4d4ba97398e6a614d965` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.222162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fa74ebad4f4d4ba97398e6a614d965)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.226120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.240020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.240773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.241064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.241250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.242001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.244163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662caff-3d92-4cf4-b95b-c4e024eff3b1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.266493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1113/2221] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:36 2022] POST /identity/v3/users => generated 299 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:37.267821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5c40-d620-4d25-bf09-66b1991df05a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.268112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5c40-d620-4d25-bf09-66b1991df05a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.268389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad5c40-d620-4d25-bf09-66b1991df05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.288686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b5ad5c40-d620-4d25-bf09-66b1991df05a None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:37.290081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b5ad5c40-d620-4d25-bf09-66b1991df05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.312468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fa74ebad4f4d4ba97398e6a614d965', 'request_id': 'req-c1d7046a-9cd0-4095-b596-dedc66b87316'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.324325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fa74ebad4f4d4ba97398e6a614d965', 'request_id': 'req-c1d7046a-9cd0-4095-b596-dedc66b87316'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.334801 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fa74ebad4f4d4ba97398e6a614d965', 'request_id': 'req-c1d7046a-9cd0-4095-b596-dedc66b87316'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.356482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] The token cache is being invalidate because project 35fa74ebad4f4d4ba97398e6a614d965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:37.357376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.358170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:37.360575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1109/2222] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:37 2022] DELETE /identity/v3/projects/35fa74ebad4f4d4ba97398e6a614d965 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:37.365207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.366658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.369278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:37.383851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1114/2223] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:37.385348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.385647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.385964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.418460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:37.419544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.431116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1115/2224] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:37.431116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.431116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.466592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7046a-9cd0-4095-b596-dedc66b87316 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.472024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.472826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.473107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.473382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.474135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.475918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34f70b-4c3f-4cba-96cb-181001efa9ab admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.495309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1110/2225] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:37.499081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.499081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.499081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.530581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814c089-f1c4-4ce1-aa0d-440a70926131 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.531684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:37.533329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:37.534415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.535093 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.535655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.535941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.549259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.550889 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.554699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1111/2226] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:37.556631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.562392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1116/2227] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:37.563295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.574222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec231a-79be-47ef-ba32-8fb7af73ab1a service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.577844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241bb7d-f299-4c51-894e-22f3677c1a62 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.583610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.584544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.584967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.585257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] PATH_INFO: `/v3/projects/5b318dc72f794e4a8f93436344c7bfc8/users/4ba2459769f041dfb76ba2a8b1281193/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.590493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.592132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.592427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.592714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.593564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:37.593886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.595754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.612586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1117/2228] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:37.618875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.628892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b318dc72f794e4a8f93436344c7bfc8, user_id=4ba2459769f041dfb76ba2a8b1281193, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.632858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.633861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a623d-adc6-44b6-9328-18ff907c6b8b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.641176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.642028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.642550 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.643428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.644451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.647725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.654111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1112/2229] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:37 2022] PUT /identity/v3/projects/5b318dc72f794e4a8f93436344c7bfc8/users/4ba2459769f041dfb76ba2a8b1281193/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:37.658178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.674921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:37.695209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1118/2230] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:37 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:37.701273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.760729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77f2ee-8200-4220-9ceb-83db4462ac70 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.806346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858072-6ed9-434d-bc74-92192de4839d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.820941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.821807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.822179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.822554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.823399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.825818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.859542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1119/2231] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:37 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:37.864481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.876561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1982c3-4c14-4d72-a1f0-0b70c6d3d0ff admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.885343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.886886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.887339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.887818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.888921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.890800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.891239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.892480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.892973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.893455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.894871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.898236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.905692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1120/2232] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:37.906675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.917906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec527b72-8adf-4ba4-9c85-64c0754c67e3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.922251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1113/2233] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:37.925298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.934512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.936102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.936566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.937314 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.938378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.942225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.951405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b5fc5-120d-40f5-b74e-b1489d0bd2d2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:37.964440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:37.966798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:37.967338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:37.967983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] PATH_INFO: `/v3/projects/5b318dc72f794e4a8f93436344c7bfc8/users/4ba2459769f041dfb76ba2a8b1281193/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:37.971917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b318dc72f794e4a8f93436344c7bfc8, user_id=4ba2459769f041dfb76ba2a8b1281193, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:37.976400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1121/2234] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:37.977455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:37.985246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.995209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1114/2235] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:37 2022] PUT /identity/v3/projects/5b318dc72f794e4a8f93436344c7bfc8/users/4ba2459769f041dfb76ba2a8b1281193/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:37.996480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:37.999251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.009946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbae07-356a-46f9-b7de-93ffd6c1a287 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:38.048739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:38.091856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:38.092992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.093408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.093811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] PATH_INFO: `/v3/projects/4127b242a4bb4ab595a1fcda9036062b/users/a20adc2432e84f1ab5738624999a8b9f/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.107246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aa171-c956-40bb-bdf6-e4d656255e69 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:38.112390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:38.113298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.113581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.113903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] PATH_INFO: `/v3/projects/dbd79b1044514826a260ce190a2b50d1` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.115212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd79b1044514826a260ce190a2b50d1)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:38.116374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4127b242a4bb4ab595a1fcda9036062b, user_id=a20adc2432e84f1ab5738624999a8b9f, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:38.117463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:38.119141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0e02-cfa8-451f-8c4b-51ef2af03379 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:38.136356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1122/2236] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:37 2022] PUT /identity/v3/projects/4127b242a4bb4ab595a1fcda9036062b/users/a20adc2432e84f1ab5738624999a8b9f/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:38.146641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.147632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.148292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.180151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:38.185737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] MFA Rules not processed for user `4ba2459769f041dfb76ba2a8b1281193`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:38.187500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.190482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd79b1044514826a260ce190a2b50d1', 'request_id': 'req-2b93638d-ccac-4b30-81be-6066b3a02376'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.199544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd79b1044514826a260ce190a2b50d1', 'request_id': 'req-2b93638d-ccac-4b30-81be-6066b3a02376'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.206735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd79b1044514826a260ce190a2b50d1', 'request_id': 'req-2b93638d-ccac-4b30-81be-6066b3a02376'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.219291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] The token cache is being invalidate because project dbd79b1044514826a260ce190a2b50d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:38.219919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.220641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b93638d-ccac-4b30-81be-6066b3a02376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.222652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1115/2237] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:37 2022] DELETE /identity/v3/projects/dbd79b1044514826a260ce190a2b50d1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:38.224274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.224556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.224841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.256058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] MFA Rules not processed for user `a20adc2432e84f1ab5738624999a8b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:38.257329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.307890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1123/2238] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:38.309170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:38.311431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.349326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1116/2239] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:38.350587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:38.352618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.417727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b162114-4d58-4cf9-a780-1767cd503088 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:38.419819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d683609-f2f3-49be-abb1-bff32be2bfb0 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:38.424568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:38.425046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:38.425362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.425692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.426063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.426218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.426348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.426614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] PATH_INFO: `/v3/users/aeaad4a2166d436c8f162654285dbbea` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.428120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeaad4a2166d436c8f162654285dbbea)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:38.430126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:38.444428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:38.444428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:38.459123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1117/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:38.460752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:38.462843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.491557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:38.504940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeaad4a2166d436c8f162654285dbbea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.518221 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeaad4a2166d436c8f162654285dbbea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.519120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeaad4a2166d436c8f162654285dbbea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.544315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeaad4a2166d436c8f162654285dbbea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.553961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeaad4a2166d436c8f162654285dbbea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:38.562344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:38.563760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1124/2241] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:38 2022] DELETE /identity/v3/users/aeaad4a2166d436c8f162654285dbbea => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:38.603363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc47d5-7865-4eab-93ec-522431fa9bee service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:38.706195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:38.707695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:38.709489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:38.709489 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:38.711601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:38.825787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:38.827606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:38.847279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1118/2242] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:38.976076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:38.979045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:39.137699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34377-bad3-4add-a3a4-b191b459b6f3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:39.161576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:39.163887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:39.164378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:39.164879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] PATH_INFO: `/v3/projects/6a3e65c6c523444b9561fc94376480d3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:39.176670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a3e65c6c523444b9561fc94376480d3)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:39.180490 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:39.282782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a3e65c6c523444b9561fc94376480d3', 'request_id': 'req-c0205992-3f8f-4677-a01b-6428e24fff57'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:39.298969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a3e65c6c523444b9561fc94376480d3', 'request_id': 'req-c0205992-3f8f-4677-a01b-6428e24fff57'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:39.319011 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a3e65c6c523444b9561fc94376480d3', 'request_id': 'req-c0205992-3f8f-4677-a01b-6428e24fff57'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:39.340309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] The token cache is being invalidate because project 6a3e65c6c523444b9561fc94376480d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:39.340760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:39.341586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:39.363520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1125/2243] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:38 2022] DELETE /identity/v3/projects/6a3e65c6c523444b9561fc94376480d3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:39.696748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:39.698984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:39.768039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33438a-f5db-4904-b6cb-9019509d2554 service glance] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:39.826808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:39.827628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:39.828045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:39.828541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:39.830799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:39.863762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:39.920255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:39.921718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:39.932003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1119/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:17:44.228578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:44.230405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:44.246212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205992-3f8f-4677-a01b-6428e24fff57 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:44.252021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:44.252742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:44.253253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:44.253574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:44.255820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:44.312681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:44.338179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:44.340875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f1044-034e-4bc3-afda-3f6694d3af66 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:44.357744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1126/2245] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:46.933444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:46.935912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:47.064998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e91e1-6431-47a3-9551-a54ce23d030c service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:47.072496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:47.073860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:47.074325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:47.075040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] PATH_INFO: `/v3/users/d80433d6983b49ffbd3436a1e8d9778a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:47.076835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80433d6983b49ffbd3436a1e8d9778a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:47.079088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:47.159359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80433d6983b49ffbd3436a1e8d9778a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:47.185569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80433d6983b49ffbd3436a1e8d9778a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:47.205130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80433d6983b49ffbd3436a1e8d9778a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:47.206479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80433d6983b49ffbd3436a1e8d9778a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:47.219928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80433d6983b49ffbd3436a1e8d9778a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:47.233986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1120/2246] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:46 2022] DELETE /identity/v3/users/d80433d6983b49ffbd3436a1e8d9778a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:47.236997 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e030-a027-4636-a420-7d496edcf673 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:47.238546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e030-a027-4636-a420-7d496edcf673 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:47.239216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473e030-a027-4636-a420-7d496edcf673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:47.264944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-4473e030-a027-4636-a420-7d496edcf673 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:47.267006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4473e030-a027-4636-a420-7d496edcf673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:47.391049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1127/2247] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:47.402776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:47.406151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:47.467240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:47.485284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5b503-8efc-4862-9385-a6c7eff4691e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:47.489781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:47.490630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:47.490822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:47.491096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:47.502984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:47.503829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:47.514875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1121/2248] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:48.047354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e030-a027-4636-a420-7d496edcf673 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:48.049885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4473e030-a027-4636-a420-7d496edcf673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:48.065451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473e030-a027-4636-a420-7d496edcf673 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:48.071361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:48.076286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:48.076286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:48.076286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] PATH_INFO: `/v3/projects/af6ed150de8548149f7726755c45b5dd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:48.079987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6ed150de8548149f7726755c45b5dd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:48.082620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:48.152584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6ed150de8548149f7726755c45b5dd', 'request_id': 'req-764b28db-695d-4953-9b1d-9f2c1095a1be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.165542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6ed150de8548149f7726755c45b5dd', 'request_id': 'req-764b28db-695d-4953-9b1d-9f2c1095a1be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.173363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6ed150de8548149f7726755c45b5dd', 'request_id': 'req-764b28db-695d-4953-9b1d-9f2c1095a1be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.194727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] The token cache is being invalidate because project af6ed150de8548149f7726755c45b5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:48.195109 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.195835 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.197544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1128/2249] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:48 2022] DELETE /identity/v3/projects/af6ed150de8548149f7726755c45b5dd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:48.202396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:48.214246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f01d9-9dd4-4e8f-ba76-766fa569e4cc service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:48.294862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:48.298148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:48.298785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:48.299408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] PATH_INFO: `/v3/users/ed73bc33ca1540d7a7e6a47135e7f01c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:48.301836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed73bc33ca1540d7a7e6a47135e7f01c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:48.305279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:48.387766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed73bc33ca1540d7a7e6a47135e7f01c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.406064 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed73bc33ca1540d7a7e6a47135e7f01c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.440279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed73bc33ca1540d7a7e6a47135e7f01c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.442714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed73bc33ca1540d7a7e6a47135e7f01c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.458492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed73bc33ca1540d7a7e6a47135e7f01c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:48.475580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1122/2250] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:48 2022] DELETE /identity/v3/users/ed73bc33ca1540d7a7e6a47135e7f01c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:49.004298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:49.006140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:49.030024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b28db-695d-4953-9b1d-9f2c1095a1be admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:49.121125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:49.122095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:49.122418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:49.122703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] PATH_INFO: `/v3/projects/0d67f4bc4dcd43d5b0f54135c19686df` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:49.124114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d67f4bc4dcd43d5b0f54135c19686df)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:49.126287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:49.149264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:49.215173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d67f4bc4dcd43d5b0f54135c19686df', 'request_id': 'req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.227710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d67f4bc4dcd43d5b0f54135c19686df', 'request_id': 'req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.238834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d67f4bc4dcd43d5b0f54135c19686df', 'request_id': 'req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.259098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] The token cache is being invalidate because project 0d67f4bc4dcd43d5b0f54135c19686df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:49.259599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.260712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-b90eb3bb-3b27-45d2-bcf1-2f89313edc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.263741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1129/2251] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:49 2022] DELETE /identity/v3/projects/0d67f4bc4dcd43d5b0f54135c19686df => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:49.268372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:49.270631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:49.293607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1c6f-eea0-472d-803b-b2076ca8a8d6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:49.413157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:49.414616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:49.415199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:49.415871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] PATH_INFO: `/v3/users/4ba2459769f041dfb76ba2a8b1281193` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:49.422982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba2459769f041dfb76ba2a8b1281193)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:49.442055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:49.451469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:49.530754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba2459769f041dfb76ba2a8b1281193'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.541349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba2459769f041dfb76ba2a8b1281193'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.557939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba2459769f041dfb76ba2a8b1281193'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.558720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba2459769f041dfb76ba2a8b1281193'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.569078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba2459769f041dfb76ba2a8b1281193'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:49.577166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1123/2252] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:49 2022] DELETE /identity/v3/users/4ba2459769f041dfb76ba2a8b1281193 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:49.621236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359400a-ee48-4f52-ab95-dedfbff7736c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:49.621773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359400a-ee48-4f52-ab95-dedfbff7736c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:49.622115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359400a-ee48-4f52-ab95-dedfbff7736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:49.656634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8359400a-ee48-4f52-ab95-dedfbff7736c None None] MFA Rules not processed for user `f705392900ef4f158b3bbc6f910fd67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:49.658713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8359400a-ee48-4f52-ab95-dedfbff7736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:49.795284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1130/2253] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:50.139372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.141666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:50.242415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd121ab-029b-4125-89bd-3cf014e6c979 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.247573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.248481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.248760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.249047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] PATH_INFO: `/v3/projects/5b318dc72f794e4a8f93436344c7bfc8` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.250365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b318dc72f794e4a8f93436344c7bfc8)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.252361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.306476 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b318dc72f794e4a8f93436344c7bfc8', 'request_id': 'req-c8c8fb63-c145-4905-8a66-7faada2262b9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:50.316048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b318dc72f794e4a8f93436344c7bfc8', 'request_id': 'req-c8c8fb63-c145-4905-8a66-7faada2262b9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:50.329203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b318dc72f794e4a8f93436344c7bfc8', 'request_id': 'req-c8c8fb63-c145-4905-8a66-7faada2262b9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:50.342985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] The token cache is being invalidate because project 5b318dc72f794e4a8f93436344c7bfc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:50.343417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:50.344197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c8c8fb63-c145-4905-8a66-7faada2262b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:50.345907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1124/2254] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:50 2022] DELETE /identity/v3/projects/5b318dc72f794e4a8f93436344c7bfc8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:50.352099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.352596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.352714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.373521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:50.374767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:50.455206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1131/2255] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:50.468255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.468743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.469221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.493479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:50.494615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:50.505448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1125/2256] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:50.510368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.521134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbf286-ffab-4c41-b034-9bd50f8a2ff5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.526615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.527426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.527903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.528115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.528793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:17:50.529130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.530764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.554691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1132/2257] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:17:50.556290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.570159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:50.583743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18e94d-1c38-4584-9b0d-5e60f63bd763 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.596940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.597710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.598066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.598416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.599199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.602280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.650243 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1126/2258] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:50 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:17:50.655242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.669006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d735774-59e2-426e-ada6-c625aaf8081a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.708387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:50.751373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00481954-4541-4305-bad3-3e642751414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.752174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00481954-4541-4305-bad3-3e642751414a admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.752757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00481954-4541-4305-bad3-3e642751414a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.753105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00481954-4541-4305-bad3-3e642751414a admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.760856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00481954-4541-4305-bad3-3e642751414a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.762813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00481954-4541-4305-bad3-3e642751414a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.799559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1133/2259] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:50 2022] POST /identity/v3/users => generated 301 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:50.802716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.816093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061ae7d-3973-4af0-8cec-723cfa653edd admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.821545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.822385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.822763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.823098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.824050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.825734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.839661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1127/2260] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:17:50.844068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00481954-4541-4305-bad3-3e642751414a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.858306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00481954-4541-4305-bad3-3e642751414a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.865952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.866638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.866909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.867178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.868438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.870284 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6645c8-03c8-4628-9014-b72ef3340112 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.883959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1134/2261] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:50.889166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:50.901215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998bae2-28a8-45b3-bb31-7a35b8b1fef7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:50.912506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:50.914203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.914784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.915349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] PATH_INFO: `/v3/projects/82d9c5ea711a4608981c0058d0adc302/users/032eaa35e89f42bc8d6cf5b6c05cc3ba/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:50.944462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d9c5ea711a4608981c0058d0adc302, user_id=032eaa35e89f42bc8d6cf5b6c05cc3ba, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:50.947319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:50.968983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1128/2262] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:50 2022] PUT /identity/v3/projects/82d9c5ea711a4608981c0058d0adc302/users/032eaa35e89f42bc8d6cf5b6c05cc3ba/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:50.981188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:50.981635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:50.982616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.029318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] MFA Rules not processed for user `032eaa35e89f42bc8d6cf5b6c05cc3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:51.030570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:51.139007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1135/2263] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:51.147906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.242752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e031f-9682-4450-9ac4-74961b37361e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.249734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.250428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.250769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.251048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.251815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.253838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.286953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1129/2264] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:17:51 2022] POST /identity/v3/projects => generated 401 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:51.291778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.307706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f67919-355a-4d15-900a-7415154a1916 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.413241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.414095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.414283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.414559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.415275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.417430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.441878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1136/2265] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:17:51 2022] POST /identity/v3/users => generated 307 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:17:51.447231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.458650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dafb8d-c911-4eaf-96ad-0f7c8ad93375 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.463918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.464650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.464934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.465213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.465968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.467653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.480427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1130/2266] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:51.485692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.496686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ed352-5060-4eb3-89a8-d6876cccadad admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.501994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.503026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.503394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.503801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] PATH_INFO: `/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7/users/0ec2efad9ac1409bb0281c711bee64aa/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.521455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8131ef9bde394fb2ade4b96ed48b71e7, user_id=0ec2efad9ac1409bb0281c711bee64aa, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.524212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.539723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1137/2267] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:51 2022] PUT /identity/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7/users/0ec2efad9ac1409bb0281c711bee64aa/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:51.544631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.555558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5935-7ef4-4c32-b8e7-c7639a978d4c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.625862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.626607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.626893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.627402 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.628716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.630878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fabf6f-6065-45b0-982e-3d359c71a0f2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.652922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1131/2268] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:17:51.658449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.669316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:51.679533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a809f79-eebe-44c2-8b3b-4fcf0e9a491e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:51.690330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:51.692398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.692775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.693317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] PATH_INFO: `/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7/users/0ec2efad9ac1409bb0281c711bee64aa/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.697212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8131ef9bde394fb2ade4b96ed48b71e7, user_id=0ec2efad9ac1409bb0281c711bee64aa, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:51.702968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:51.723380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1138/2269] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:17:51 2022] PUT /identity/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7/users/0ec2efad9ac1409bb0281c711bee64aa/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:51.746545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:51.747034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:51.747533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:51.796356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] MFA Rules not processed for user `0ec2efad9ac1409bb0281c711bee64aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:51.797637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:51.810433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:51.917981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1132/2270] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:51.930214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:51.932699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:52.014971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaa8c6-c678-4615-9ca2-7f0acb1f125d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:52.019076 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:52.019870 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:52.020158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:52.020523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:52.031866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:52.033476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:52.045398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1139/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:52.148749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:52.167148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f177d2-00d8-405c-b603-a59afeefde7e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:52.171816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:52.172548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:52.172813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:52.173084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:52.299912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:52.301989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc186876-067d-45e9-a69a-06627e27353a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:52.318966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1133/2272] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:57.411463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:57.621283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fa3df-7049-4950-a74d-266ea1a0f89a service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:57.630406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:57.631790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:57.631930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:57.632145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] PATH_INFO: `/v3/users/032eaa35e89f42bc8d6cf5b6c05cc3ba` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:57.633864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032eaa35e89f42bc8d6cf5b6c05cc3ba)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:57.636072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:57.761879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032eaa35e89f42bc8d6cf5b6c05cc3ba'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:57.780193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032eaa35e89f42bc8d6cf5b6c05cc3ba'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:57.783216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032eaa35e89f42bc8d6cf5b6c05cc3ba'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:57.816320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032eaa35e89f42bc8d6cf5b6c05cc3ba'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:57.827380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032eaa35e89f42bc8d6cf5b6c05cc3ba'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:57.871426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1140/2273] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:57 2022] DELETE /identity/v3/users/032eaa35e89f42bc8d6cf5b6c05cc3ba => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:57.873823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:57.874610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:57.874610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:57.916274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:17:57.917791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:58.078859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1134/2274] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:17:58.114168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:58.115871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:58.133264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:58.252369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86f57a-0fc9-4776-979b-34f0f712c3d8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:58.265911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:58.268346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:58.269133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:58.269133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:58.280675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:58.282491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:58.296619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1141/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:17:58.966066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:58.969329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:17:58.986424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd3d72-fb04-4c98-af14-460f987b8d7d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:58.991875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:58.992800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:58.993086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:58.993393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] PATH_INFO: `/v3/projects/82d9c5ea711a4608981c0058d0adc302` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:58.994820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d9c5ea711a4608981c0058d0adc302)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:58.996910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:59.064706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:17:59.085517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d9c5ea711a4608981c0058d0adc302', 'request_id': 'req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.098901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d9c5ea711a4608981c0058d0adc302', 'request_id': 'req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.109509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d9c5ea711a4608981c0058d0adc302', 'request_id': 'req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.126104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] The token cache is being invalidate because project 82d9c5ea711a4608981c0058d0adc302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:17:59.126404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.127336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.129290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1135/2276] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:58 2022] DELETE /identity/v3/projects/82d9c5ea711a4608981c0058d0adc302 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:17:59.134681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:59.146986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f6ff-ace0-49cf-91db-2dd748f43238 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:17:59.270867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:17:59.273114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:17:59.273884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:17:59.274556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] PATH_INFO: `/v3/users/0ec2efad9ac1409bb0281c711bee64aa` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:17:59.278062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec2efad9ac1409bb0281c711bee64aa)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:17:59.283967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:17:59.376300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec2efad9ac1409bb0281c711bee64aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.392814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec2efad9ac1409bb0281c711bee64aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.394106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec2efad9ac1409bb0281c711bee64aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.425297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec2efad9ac1409bb0281c711bee64aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.442332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-51467f69-92aa-4c46-84ba-3acab78f5148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec2efad9ac1409bb0281c711bee64aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:17:59.452091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1142/2277] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:17:59 2022] DELETE /identity/v3/users/0ec2efad9ac1409bb0281c711bee64aa => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:17:59.972263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:17:59.979637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:00.012727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c9-7fac-4b2a-bf12-560f949d8b4e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.094404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.095344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.095843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.096304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] PATH_INFO: `/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.097663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8131ef9bde394fb2ade4b96ed48b71e7)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.099699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.166905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8131ef9bde394fb2ade4b96ed48b71e7', 'request_id': 'req-2f8cc34c-2936-420d-89c8-4cf79d963a50'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:00.178166 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8131ef9bde394fb2ade4b96ed48b71e7', 'request_id': 'req-2f8cc34c-2936-420d-89c8-4cf79d963a50'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:00.187156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8131ef9bde394fb2ade4b96ed48b71e7', 'request_id': 'req-2f8cc34c-2936-420d-89c8-4cf79d963a50'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:00.200604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] The token cache is being invalidate because project 8131ef9bde394fb2ade4b96ed48b71e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:00.200970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:00.201796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2f8cc34c-2936-420d-89c8-4cf79d963a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:00.203624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1136/2278] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:17:59 2022] DELETE /identity/v3/projects/8131ef9bde394fb2ade4b96ed48b71e7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:00.209130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.209681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.209771 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.229994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:00.231269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:00.267326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:00.323182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1143/2279] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:00.333214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.333777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.334228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.361607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:00.363406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:00.375121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1137/2280] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:00.379912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.393789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d847e9-6e33-4796-95da-c864b3ca5aa3 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.399787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.400596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.401000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.401426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.402452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:18:00.402887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.404772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.419946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1144/2281] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:18:00.425454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.438464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c14fa2-3975-41c1-b38c-e106137c7c03 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.445475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.446535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.446862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.447176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.448013 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.451035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.491001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1138/2282] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:00 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:00.496689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.512601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0a91-f277-4a79-9630-46aaaa350758 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.633601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.634799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.635423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.636007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.637428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.641831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.688871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1145/2283] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:00 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:00.689188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.701809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200361-a167-43dc-8eac-1fe16d966d20 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.708343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.709127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.709605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.710237 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.711809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.714250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.729817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1139/2284] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:18:00.734996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.748937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923abb-0887-4ab0-bfdd-bb2cccac3ce7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.758813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.760448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.760960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.761382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.762772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.765721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49c72eb-7033-4113-b987-db5a7e82f98d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.780178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1146/2285] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:18:00.786741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:00.801636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b51c-9806-43f1-ae90-aee94d2d660e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:00.807443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:00.808463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.808751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.809075 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] PATH_INFO: `/v3/projects/424c13e60aff4809ad9df53e657fef0a/users/899c9b338ee342bc8d441f6a0c751d04/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.826459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:00.831397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] RBAC: Authorizing `identity:create_grant(project_id=424c13e60aff4809ad9df53e657fef0a, user_id=899c9b338ee342bc8d441f6a0c751d04, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:00.835078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:00.859182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1140/2286] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:00 2022] PUT /identity/v3/projects/424c13e60aff4809ad9df53e657fef0a/users/899c9b338ee342bc8d441f6a0c751d04/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:00.870957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:00.871327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:00.871718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:00.902589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] MFA Rules not processed for user `899c9b338ee342bc8d441f6a0c751d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:00.904009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:01.016853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1147/2287] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:01.023843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.149642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feed4d0-e612-4b90-89dc-c40aca8c709f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.160079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.163305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.164662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.165179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.167699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.170561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.228153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1141/2288] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:01 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:01.232836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.246668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e49ae1-fe46-4ae7-90d8-3a4b98137be2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.310215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:01.388262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.389488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.391071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.391071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.392291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.397125 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.435530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1148/2289] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:01 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:01.439685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.460354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867c3d-dce4-411e-b137-63ba940ca00b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.466831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.468560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.469060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.469712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.470771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.473469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.496390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1142/2290] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:01.502329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.520372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd20a7-4b87-42d0-8bdf-41c107c02665 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.528517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.530229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.530820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.531927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] PATH_INFO: `/v3/projects/a28c2dd0cb0f483b8180d9781486c42d/users/c1f10eed6e054744aeba456f99e0062b/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.559233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28c2dd0cb0f483b8180d9781486c42d, user_id=c1f10eed6e054744aeba456f99e0062b, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.562182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.590708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1149/2291] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:01 2022] PUT /identity/v3/projects/a28c2dd0cb0f483b8180d9781486c42d/users/c1f10eed6e054744aeba456f99e0062b/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:01.601188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.614354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7490b-7386-42e2-8d24-744f012224c0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.720759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.722015 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.722567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.723054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.724571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.727702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea1959-511a-42c7-ab86-9ce7497a0e01 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.748680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1143/2292] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:01.755130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:01.773874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b60cd-39f2-45d5-ba66-a0ae635f182c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:01.779469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:01.780410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.780689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.780966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] PATH_INFO: `/v3/projects/a28c2dd0cb0f483b8180d9781486c42d/users/c1f10eed6e054744aeba456f99e0062b/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.784456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28c2dd0cb0f483b8180d9781486c42d, user_id=c1f10eed6e054744aeba456f99e0062b, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:01.795694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:01.833043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1150/2293] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:01 2022] PUT /identity/v3/projects/a28c2dd0cb0f483b8180d9781486c42d/users/c1f10eed6e054744aeba456f99e0062b/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:18:01.854914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987fbb2-b9f7-4210-b9df-5acd609ae8a7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:01.855305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987fbb2-b9f7-4210-b9df-5acd609ae8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:01.855771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987fbb2-b9f7-4210-b9df-5acd609ae8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:01.937584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-6987fbb2-b9f7-4210-b9df-5acd609ae8a7 None None] MFA Rules not processed for user `c1f10eed6e054744aeba456f99e0062b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:01.939876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6987fbb2-b9f7-4210-b9df-5acd609ae8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:02.052250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:02.139702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1144/2294] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:02.169610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:02.175526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:02.308243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84d54-e1c4-470c-8cc6-36ef83609865 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:02.314633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:02.316370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:02.316901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:02.317418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:02.335514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:02.338250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:02.369952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1151/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:02.505773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:02.506268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:02.506766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:02.545550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] MFA Rules not processed for user `a14ddfab91e94f6db6d7f6596ca31916`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:02.548685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:02.574482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:02.592735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257541-f2e7-4449-8101-216922ce3c60 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:02.615890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:02.617056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:02.617637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:02.618381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:02.678653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1145/2296] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:02.731969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:02.748300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399c4f7-6c07-4b36-a177-963d6f20943c service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:02.758051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1152/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:08.903510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:09.020894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73014d-f939-4276-93f6-c6f6d345d3c4 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:09.036344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:09.036344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:09.036344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:09.036344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] PATH_INFO: `/v3/users/899c9b338ee342bc8d441f6a0c751d04` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:09.037317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899c9b338ee342bc8d441f6a0c751d04)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:09.037317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:09.166422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899c9b338ee342bc8d441f6a0c751d04'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:09.181495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899c9b338ee342bc8d441f6a0c751d04'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:09.222128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899c9b338ee342bc8d441f6a0c751d04'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:09.223591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899c9b338ee342bc8d441f6a0c751d04'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:09.235078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899c9b338ee342bc8d441f6a0c751d04'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:09.251578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1146/2298] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:18:08 2022] DELETE /identity/v3/users/899c9b338ee342bc8d441f6a0c751d04 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:09.254416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:09.254975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:09.255367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:09.283926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:09.286282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:09.334488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:09.493334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1153/2299] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:09.507304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:09.509894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:09.572589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:09.830254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e5409-5136-451f-8d50-2e832351c8b5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:09.854268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:09.857425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:09.857959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:09.858505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:09.906609 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:09.911686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:09.938431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1147/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:10.472501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:10.478567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:10.546019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69427c46-646a-47df-a061-7a28c8d89016 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:10.559669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:10.561694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:10.562530 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:10.563967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] PATH_INFO: `/v3/projects/424c13e60aff4809ad9df53e657fef0a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:10.567776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=424c13e60aff4809ad9df53e657fef0a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:10.571842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:10.734398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '424c13e60aff4809ad9df53e657fef0a', 'request_id': 'req-4111348c-430f-466c-a9df-3c4eea7381c5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:10.743092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '424c13e60aff4809ad9df53e657fef0a', 'request_id': 'req-4111348c-430f-466c-a9df-3c4eea7381c5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:10.754994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '424c13e60aff4809ad9df53e657fef0a', 'request_id': 'req-4111348c-430f-466c-a9df-3c4eea7381c5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:10.784553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] The token cache is being invalidate because project 424c13e60aff4809ad9df53e657fef0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:10.785064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:10.786449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:10.789171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1154/2301] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:18:10 2022] DELETE /identity/v3/projects/424c13e60aff4809ad9df53e657fef0a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:10.794899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:10.818753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42285e-e316-4a02-a63e-cd1f302bcc28 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:10.916650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:10.917965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:10.918155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:10.918416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] PATH_INFO: `/v3/users/c1f10eed6e054744aeba456f99e0062b` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:10.920164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f10eed6e054744aeba456f99e0062b)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:10.922492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:11.056181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f10eed6e054744aeba456f99e0062b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:11.075669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f10eed6e054744aeba456f99e0062b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:11.129761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f10eed6e054744aeba456f99e0062b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:11.131313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f10eed6e054744aeba456f99e0062b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:11.160639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-05f9a81f-85e2-4c96-82a1-38d0a21a0441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f10eed6e054744aeba456f99e0062b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:11.188687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1148/2302] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:18:10 2022] DELETE /identity/v3/users/c1f10eed6e054744aeba456f99e0062b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:12.586450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:12.588428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:12.614261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111348c-430f-466c-a9df-3c4eea7381c5 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:12.727909 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:12.730581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:12.732144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:12.733255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] PATH_INFO: `/v3/projects/a28c2dd0cb0f483b8180d9781486c42d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:12.735975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28c2dd0cb0f483b8180d9781486c42d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:12.740542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:12.833992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:12.836978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28c2dd0cb0f483b8180d9781486c42d', 'request_id': 'req-79d21197-61d2-4d44-ba20-047ba4dd0aa3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:12.848802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28c2dd0cb0f483b8180d9781486c42d', 'request_id': 'req-79d21197-61d2-4d44-ba20-047ba4dd0aa3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:12.855897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28c2dd0cb0f483b8180d9781486c42d', 'request_id': 'req-79d21197-61d2-4d44-ba20-047ba4dd0aa3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:12.872760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] The token cache is being invalidate because project a28c2dd0cb0f483b8180d9781486c42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:12.873250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:12.874112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-79d21197-61d2-4d44-ba20-047ba4dd0aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:12.876133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1155/2303] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:18:12 2022] DELETE /identity/v3/projects/a28c2dd0cb0f483b8180d9781486c42d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:12.881502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:12.882096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:12.882178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:12.906162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:12.907934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:12.960712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:13.021146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1149/2304] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:13.031246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.031575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.031849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.058817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:13.061205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:13.078183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1156/2305] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:13.083376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.110180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa52f81-544b-4a70-8af4-ffad8ff4147b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.118955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.120351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.121053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.121561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.123776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:18:13.124455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.127938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.152652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1150/2306] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:18:13.162757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.186611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec1e29-c832-4bd6-8c73-3bc99d683270 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.205635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.206877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.207470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.207967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.211618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.215522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.266920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1157/2307] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:13 2022] POST /identity/v3/projects => generated 353 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:13.274439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.299271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365682d-d732-468e-a49c-5f11bcb7c114 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.421872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.423183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.423795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.424372 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.425960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.429919 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.481871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1151/2308] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:13 2022] POST /identity/v3/users => generated 277 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:18:13.485178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.504781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bd292-c33a-4aef-8611-b6235ac70a7d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.513814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.515357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.516138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.516933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.518740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.523728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.544075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1158/2309] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:13.549363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.570878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33ece-7eb3-4d9b-9cb9-9c21b9bc3fe2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.579751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.580991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.581554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.583549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.583549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.586491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b942f5de-e87f-4425-9f81-42da4a5b8074 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.607549 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1152/2310] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:18:13.613139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.631666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3d48-cb91-43c4-936d-ad9dce04567a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:13.642576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:13.644564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.645334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.645900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] PATH_INFO: `/v3/projects/8f0286ae5ec749259803003b4fc98810/users/1b2684f9dae74173a2e03a721c4b8bb4/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.675217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0286ae5ec749259803003b4fc98810, user_id=1b2684f9dae74173a2e03a721c4b8bb4, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:13.680500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:13.711115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1159/2311] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:13 2022] PUT /identity/v3/projects/8f0286ae5ec749259803003b4fc98810/users/1b2684f9dae74173a2e03a721c4b8bb4/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:13.721431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:13.721777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:13.721942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:13.780285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] MFA Rules not processed for user `1b2684f9dae74173a2e03a721c4b8bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:13.782633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:13.916370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1153/2312] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:13.965402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:13.967885 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:14.061126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d766bb-33a4-40ac-af67-c080123150f6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:14.067675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:14.068827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:14.069292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:14.069755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:14.085598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:14.089036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c06c0-1ce1-4b12-acda-74e83536824f service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:14.106829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1160/2313] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:14.125589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:14.173726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:14.215954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f328650-f1a0-433f-aac3-4a11c6519354 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:14.222167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:14.223338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:14.223756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:14.224131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] PATH_INFO: `/v3/users/1b2684f9dae74173a2e03a721c4b8bb4` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:14.225765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2684f9dae74173a2e03a721c4b8bb4)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:14.228108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:14.304215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2684f9dae74173a2e03a721c4b8bb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:14.317228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2684f9dae74173a2e03a721c4b8bb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:14.336407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2684f9dae74173a2e03a721c4b8bb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:14.337191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2684f9dae74173a2e03a721c4b8bb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:14.347716 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2684f9dae74173a2e03a721c4b8bb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:14.356403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1154/2314] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:18:14 2022] DELETE /identity/v3/users/1b2684f9dae74173a2e03a721c4b8bb4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:14.361427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:14.361798 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:14.362156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:14.383739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:14.385464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:14.478972 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1161/2315] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:14.490161 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:14.491928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:14.605752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ff481-bbb0-4aa1-a6ca-780328c37e94 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:14.612123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:14.612938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:14.613287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:14.613642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:14.626441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:14.628366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e2552-9f54-4d77-b750-8d5c2d50952a service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:14.640901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1155/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:18:15.053282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.055690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:15.074206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175021cf-1803-421e-83cd-2e32f67a9150 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.081135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.082534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.083065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.083762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] PATH_INFO: `/v3/projects/8f0286ae5ec749259803003b4fc98810` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.085552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0286ae5ec749259803003b4fc98810)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.088165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.101485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:15.169046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0286ae5ec749259803003b4fc98810', 'request_id': 'req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:15.182139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0286ae5ec749259803003b4fc98810', 'request_id': 'req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:15.198564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0286ae5ec749259803003b4fc98810', 'request_id': 'req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:15.218423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] The token cache is being invalidate because project 8f0286ae5ec749259803003b4fc98810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:15.218856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:15.220017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e0ac79d5-05ca-4fd7-af69-c2d0ca2980f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:15.222514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1162/2317] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:18:15 2022] DELETE /identity/v3/projects/8f0286ae5ec749259803003b4fc98810 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:18:15.228399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.229304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.229304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.264907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:15.266873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:15.353599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:15.375340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1156/2318] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:15.384000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.384408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.384787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.406623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:15.407970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:15.418135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1163/2319] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:15.424412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.438605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76564c-2011-45fd-b30d-85dd9898a2b3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.445141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.446029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.446401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.446779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.447690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:18:15.448214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.450009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.465363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1157/2320] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:18:15.470777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.481915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4e832-e4fc-4bc0-ad30-ada98e08617b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.488448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.489158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.489448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.489762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.490438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.492595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.525517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1164/2321] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:18:15.530721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.543168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456ad0-b83d-4386-9401-f57c495950c8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.621570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.622281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.622560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.622909 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.623669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.625732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.657296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1158/2322] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:15 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:15.662601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.676467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562315a5-1c72-431a-8ab6-2b95a65098eb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.685234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.686073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.686450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.686853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.687887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.690850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.709183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1165/2323] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:15.714653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.731885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911c4e-a4cf-4229-89f2-21e49a428d49 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.737879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.739197 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.739657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.740081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.742009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.744089 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9ba7-3092-4c6e-b98f-aeb7557c383e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.763772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1159/2324] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:15.765932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:15.792702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac267-ea42-4869-bbd2-61c60f219a9e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:15.800486 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:15.801884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.802348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.802735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] PATH_INFO: `/v3/projects/6f45498ad3db419f8d5a5dd12088893f/users/74a829a1020b4f79821bb07d3eb95f8c/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.828610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f45498ad3db419f8d5a5dd12088893f, user_id=74a829a1020b4f79821bb07d3eb95f8c, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:15.833342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:15.857181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1166/2325] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:15 2022] PUT /identity/v3/projects/6f45498ad3db419f8d5a5dd12088893f/users/74a829a1020b4f79821bb07d3eb95f8c/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:15.874984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f78a3-7cd8-4bed-8230-1987eda9a2b9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:15.875724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f78a3-7cd8-4bed-8230-1987eda9a2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:15.876283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f78a3-7cd8-4bed-8230-1987eda9a2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:15.914468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-358f78a3-7cd8-4bed-8230-1987eda9a2b9 None None] MFA Rules not processed for user `74a829a1020b4f79821bb07d3eb95f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:15.916046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-358f78a3-7cd8-4bed-8230-1987eda9a2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:16.023761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1160/2326] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:16.037932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:16.040050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:16.110869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f573a-efd4-49a9-a8df-42c558c2833a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:16.115238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:16.115966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:16.116250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:16.116533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:16.128784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:16.131725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:16.167964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1167/2327] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:18:17.792530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:17.793446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:17.793607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:17.827590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:17.829786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:17.922145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:17.967407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1161/2328] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:18:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:18.047551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:18.049605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:18.120017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:18.172021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2b0ec-f720-47b9-a059-000c61467b99 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:18.179073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:18.179863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:18.180244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:18.180765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:18.192971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:18.194713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617a1d2-7d50-47ba-a883-df9eaca01dd4 service swift] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:18.204930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1168/2329] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:18:26.939680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:26.945405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:27.079198 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121643ee-f4f7-46ab-8b2e-bfd5d83c5402 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:27.100864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:27.101940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:27.102322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:27.102694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] PATH_INFO: `/v3/users/a20adc2432e84f1ab5738624999a8b9f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:27.106103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20adc2432e84f1ab5738624999a8b9f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:27.108297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:27.330654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20adc2432e84f1ab5738624999a8b9f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:27.346667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20adc2432e84f1ab5738624999a8b9f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:27.361637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20adc2432e84f1ab5738624999a8b9f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:27.362405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20adc2432e84f1ab5738624999a8b9f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:27.370512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20adc2432e84f1ab5738624999a8b9f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:27.377813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1162/2330] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:18:26 2022] DELETE /identity/v3/users/a20adc2432e84f1ab5738624999a8b9f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:27.382965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:27.384002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:27.384002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:27.406014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:27.408545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:27.494122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1169/2331] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:27.509138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:27.512422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:27.635426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac7b68-b888-4624-b701-f25954406ec8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:27.640057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:27.642053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:27.642053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:27.642401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:27.655858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:27.657469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:27.673864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1163/2332] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:27.849706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:27.854279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:27.890418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9070b-eace-4474-9b2d-3d60dd2fb04c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:27.898545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:27.899749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:27.900245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:27.900796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:27.905217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:28.068468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:28.070214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dac93e-040f-4602-a47a-2e3e2e4b762d service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:28.101309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1170/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:28.524433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:28.527509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:28.563933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20145589-3186-4706-93ee-d17b97f51990 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:28.584530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:28.586046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:28.586477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:28.586785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] PATH_INFO: `/v3/projects/4127b242a4bb4ab595a1fcda9036062b` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:28.588281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4127b242a4bb4ab595a1fcda9036062b)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:28.590346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:28.612435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:28.692017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4127b242a4bb4ab595a1fcda9036062b', 'request_id': 'req-2962cb44-a8de-4695-a5bd-827ca42ded12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:28.706674 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4127b242a4bb4ab595a1fcda9036062b', 'request_id': 'req-2962cb44-a8de-4695-a5bd-827ca42ded12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:28.719009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4127b242a4bb4ab595a1fcda9036062b', 'request_id': 'req-2962cb44-a8de-4695-a5bd-827ca42ded12'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:28.740276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] The token cache is being invalidate because project 4127b242a4bb4ab595a1fcda9036062b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:28.740615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:28.741887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2962cb44-a8de-4695-a5bd-827ca42ded12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:28.748680 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1164/2334] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:18:28 2022] DELETE /identity/v3/projects/4127b242a4bb4ab595a1fcda9036062b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:28.750352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:28.750894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:28.751264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:28.824580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:28.826617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:29.038286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1171/2335] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:29.050612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.050906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.051546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.092300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:29.097886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:29.113991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1165/2336] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:29.115135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.121364 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:29.130987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4181-aaae-441b-8363-5716021ef1b0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.137194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.139222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.139575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.140325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.141119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:18:29.141667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.143922 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.168240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1172/2337] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:18:29.170153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.207994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320338ee-9e7d-41c9-ba72-2b8bd782542d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.260336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.261201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.261606 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.262298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.262815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.271546 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.366648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1166/2338] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:29 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:29.413080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.453783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3ce1f-ca23-47b0-9b88-d474a1b449f0 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.634500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.635706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.636167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.637160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.638091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.641307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.674864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1173/2339] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:29 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:29.679577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.704276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9706704-50a6-43a6-8607-bf799f53326b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.711189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.712567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.712567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.712567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.713700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.715451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.737017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1167/2340] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:18:29.737993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.756431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5cb32-25ca-451b-bddf-d60a3e27d135 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.767751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.768116 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.770535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.770535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.771227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.775313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c622a-582f-4076-bbc0-53d757c22346 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.807182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1174/2341] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:18:29.808528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:29.821979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38439f-df56-4151-bf27-c187a5b956cd admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:29.830875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:29.832253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.832478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.832815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] PATH_INFO: `/v3/projects/53de01ad2f474a2fbd9c37598c1cee6a/users/c7bdd8ec6d4947ee9e6f9e963a6f4efb/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.853409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53de01ad2f474a2fbd9c37598c1cee6a, user_id=c7bdd8ec6d4947ee9e6f9e963a6f4efb, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:29.856844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:29.894033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1168/2342] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:29 2022] PUT /identity/v3/projects/53de01ad2f474a2fbd9c37598c1cee6a/users/c7bdd8ec6d4947ee9e6f9e963a6f4efb/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:29.896776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:29.897190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:29.897824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:29.945303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] MFA Rules not processed for user `c7bdd8ec6d4947ee9e6f9e963a6f4efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:29.946611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:30.048149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1175/2343] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:30.055750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:30.057629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:30.175906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012ed88-dde2-4998-80fe-03c509d8b407 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:30.183635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:30.184765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:30.185061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:30.185343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:30.216654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:30.218667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:30.239726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1169/2344] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:38.341050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:38.342561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:38.412484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:38.430713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb05f1d-f0fe-4400-944b-d1f7148caffa None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:38.440217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:38.440895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:38.441175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:38.441448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:38.443427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:38.456380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:38.457910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f07d97-2aae-4847-b26a-67a6b530f264 service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:38.470000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1176/2345] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:38.955850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:38.957493 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.013415 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:39.046893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685bfa8-97d8-49cc-893b-882fcdd43d4f service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:39.053583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:39.054496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.054786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.055070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] PATH_INFO: `/v3/users/74a829a1020b4f79821bb07d3eb95f8c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.056411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a829a1020b4f79821bb07d3eb95f8c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:39.058393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:39.115374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a829a1020b4f79821bb07d3eb95f8c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:39.124843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a829a1020b4f79821bb07d3eb95f8c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:39.140122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a829a1020b4f79821bb07d3eb95f8c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:39.140950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a829a1020b4f79821bb07d3eb95f8c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:39.150902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a829a1020b4f79821bb07d3eb95f8c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:39.159263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1170/2346] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:18:38 2022] DELETE /identity/v3/users/74a829a1020b4f79821bb07d3eb95f8c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:39.169167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0263f-79ac-4ae9-909b-4e9248214e71 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.169167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0263f-79ac-4ae9-909b-4e9248214e71 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.169167 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0263f-79ac-4ae9-909b-4e9248214e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.189418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0db0263f-79ac-4ae9-909b-4e9248214e71 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:39.191098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0db0263f-79ac-4ae9-909b-4e9248214e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.267381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1177/2347] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:39.277320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:39.279286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.356313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e17430-9d38-47d4-afcc-da03848bcfd4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:39.361097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:39.361796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.362061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.362331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.373511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:39.374993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35419-4b19-4083-aa67-ae1a8966e7eb service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:39.393724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1171/2348] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:39.709253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.709902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.709902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.711982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1178/2349] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:18:39.722118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.722509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.722854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.744666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:39.746043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.824017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:39.857381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1172/2350] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:39.870072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:39.872414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.892869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:39.897467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:39.920950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b6339-ef98-46ec-9e89-563e25120305 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:39.928510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:39.928927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.929262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.929420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] PATH_INFO: `/v3/projects/6f45498ad3db419f8d5a5dd12088893f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:39.930741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f45498ad3db419f8d5a5dd12088893f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:39.933134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:39.989773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1c147-1598-414a-a45c-2890a927c747 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:39.993994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:39.994917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:39.995429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:39.995873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.009420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.011308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8c2c2-b48e-4ebb-ab9c-c4e309373b62 service glance] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.021982 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f45498ad3db419f8d5a5dd12088893f', 'request_id': 'req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:40.026552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1179/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:40.033521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f45498ad3db419f8d5a5dd12088893f', 'request_id': 'req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:40.041502 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f45498ad3db419f8d5a5dd12088893f', 'request_id': 'req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:40.056233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] The token cache is being invalidate because project 6f45498ad3db419f8d5a5dd12088893f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:18:40.056506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:40.057434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ebbf5744-c1ce-4eae-bc0d-d7fc4fcf7c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:18:40.059004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1173/2352] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:18:39 2022] DELETE /identity/v3/projects/6f45498ad3db419f8d5a5dd12088893f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:40.064070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b24cf6-8b26-4468-af29-ba95e4765462 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.064960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b24cf6-8b26-4468-af29-ba95e4765462 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.065401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b24cf6-8b26-4468-af29-ba95e4765462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.086907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-93b24cf6-8b26-4468-af29-ba95e4765462 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:40.088331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-93b24cf6-8b26-4468-af29-ba95e4765462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.111146 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420bbd-1db7-4304-8a4f-4bd5bcc369e4 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.111723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420bbd-1db7-4304-8a4f-4bd5bcc369e4 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.112002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420bbd-1db7-4304-8a4f-4bd5bcc369e4 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.113481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1174/2353] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 12:18:40.118800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.119632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.119632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.139904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:40.141147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.160427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1180/2354] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:40.169333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.169622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.169894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.199361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:40.200721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.209738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1181/2355] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:40.214990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.225686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4966c4-5c55-4456-99b5-bb8e8dd5ea79 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.226616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1175/2356] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:40.232520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.233352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.233770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.234105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.235104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:18:40.235556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.237356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.238599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.240442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.250687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1182/2357] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:18:40.251998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bfeda-e5fe-493d-970e-fd68e16dd5b1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.255525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.256574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.257331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.257708 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.258058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.268008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043ae78-2523-442a-9df5-0a76a4fdae45 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.270448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.272116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.273302 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.274006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.274286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.274558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.275264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.277282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be122e0-c547-44cc-9daa-cfcfa1f1ad6b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.282134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1176/2358] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:18:40.286937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:40.310691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1183/2359] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:40.312274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.326706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d2c4-dfc2-454d-a938-33185b711e96 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.368634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a368-1ce2-44f3-b7f4-4ae8141dfbbb None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.368896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a368-1ce2-44f3-b7f4-4ae8141dfbbb None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.369191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a368-1ce2-44f3-b7f4-4ae8141dfbbb None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.370740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1184/2360] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:18:40.376197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.376660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.377003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.391101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.391844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.392126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.392399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.393152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.395256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.396058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:40.397293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.412416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:40.426652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1177/2361] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:18:40.427665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.438380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346a00e-ff02-4632-b074-ddeaaefe84fa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.444648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.445756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.445756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.445756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.446600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.448279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.463397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1178/2362] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:40.469239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.485261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d29cc-e030-4c1a-9689-42bc02d7c2fd admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.486846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1185/2363] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:40.492888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.494049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.495041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.495041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.495964 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.497803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.501425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.502882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.529428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1179/2364] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:18:40.531721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.546749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2441b-4b5c-4b68-a28b-1c3cb8b506f0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.552310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.553259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.553540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.553815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] PATH_INFO: `/v3/projects/4a3a3d9d22d04ab490a4ef62c0a87187/users/e4c5acb8ae014c8a9efc303122675038/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.573522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3a3d9d22d04ab490a4ef62c0a87187, user_id=e4c5acb8ae014c8a9efc303122675038, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.574431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb1ab-6415-4f8c-baee-74f606baf92d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.576352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e77dab-52b7-4671-b2c1-82017c63b707 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.579233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.580394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.581072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.581698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.594087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1180/2365] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:18:40 2022] PUT /identity/v3/projects/4a3a3d9d22d04ab490a4ef62c0a87187/users/e4c5acb8ae014c8a9efc303122675038/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:18:40.594296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.595984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.605686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507164f-c17a-483e-923a-63ea7190c6ee None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.606063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507164f-c17a-483e-923a-63ea7190c6ee None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.606329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507164f-c17a-483e-923a-63ea7190c6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.608816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1186/2366] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:40.636794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-3507164f-c17a-483e-923a-63ea7190c6ee None None] MFA Rules not processed for user `e4c5acb8ae014c8a9efc303122675038`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:40.638345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3507164f-c17a-483e-923a-63ea7190c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.723384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1181/2367] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:18:40.736693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:40.738217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:40.800527 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58371c34-52dd-4009-a3c9-fcdd2f8c0270 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:40.844899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:40.859007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:40.859708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:40.860127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:40.860401 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:40.869759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:40.871428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:40.880709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1187/2368] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:18:46.565389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:46.565972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:46.566471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:46.589830 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:18:46.591152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:46.656034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1182/2369] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:18:46.669463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:18:46.671226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:18:46.746983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d841a2-5503-4020-ad73-960e03c265d6 service glance] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:18:46.752715 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:18:46.753395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:18:46.753674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:18:46.753950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:18:46.764372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:18:46.766127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e11a7b-8cdb-46cd-9377-39995cb4511f service swift] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:18:46.776808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1188/2370] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:18:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:18:56.648555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:18:56.772603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=40 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:16.578736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:16.678139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1d60-df9c-48f3-b6c6-873059ed182f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:16.683412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:16.684337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:16.684622 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:16.684902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] PATH_INFO: `/v3/users/e4c5acb8ae014c8a9efc303122675038` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:16.686130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c5acb8ae014c8a9efc303122675038)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:16.688245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:16.750802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c5acb8ae014c8a9efc303122675038'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:16.759819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c5acb8ae014c8a9efc303122675038'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:16.775897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c5acb8ae014c8a9efc303122675038'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:16.776768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c5acb8ae014c8a9efc303122675038'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:16.785188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c5acb8ae014c8a9efc303122675038'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:16.795996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1183/2371] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:19:16 2022] DELETE /identity/v3/users/e4c5acb8ae014c8a9efc303122675038 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:16.800876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:16.800876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:16.801383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:16.822115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:16.823474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:16.889520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1189/2372] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:19:16.900286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:16.902125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:16.970170 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c8987-7ba2-47c3-a92d-82473a9f8344 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:16.974196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:16.974893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:16.975189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:16.975463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:16.987138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:16.988722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c2222-e67c-455f-b375-c60f40441a03 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.003040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1184/2373] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:19:17.353695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.355521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:17.376790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a577bd-c563-4e9a-884e-b60d18e18121 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.393452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.394349 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.394627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.394899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] PATH_INFO: `/v3/projects/4a3a3d9d22d04ab490a4ef62c0a87187` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.396101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3a3d9d22d04ab490a4ef62c0a87187)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.399190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.449897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3a3d9d22d04ab490a4ef62c0a87187', 'request_id': 'req-9cd381ca-1282-4695-aaf4-e97ce788e883'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:17.457662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3a3d9d22d04ab490a4ef62c0a87187', 'request_id': 'req-9cd381ca-1282-4695-aaf4-e97ce788e883'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:17.465679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3a3d9d22d04ab490a4ef62c0a87187', 'request_id': 'req-9cd381ca-1282-4695-aaf4-e97ce788e883'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:17.481035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] The token cache is being invalidate because project 4a3a3d9d22d04ab490a4ef62c0a87187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:19:17.481610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:17.482311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9cd381ca-1282-4695-aaf4-e97ce788e883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:17.487631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1190/2374] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:19:17 2022] DELETE /identity/v3/projects/4a3a3d9d22d04ab490a4ef62c0a87187 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:17.489722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.490011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.490290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.509330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:17.510577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:17.530364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:17.588663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1185/2375] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:17.597482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.597772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.598056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.617770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:17.619262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:17.633952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1191/2376] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:17.635226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.645747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c0e5e-91a6-4809-9c9a-f5d087fa064e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.650987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.651718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.652002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.652273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.653018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:19:17.653322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.655182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.668649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1186/2377] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:19:17.673274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.684188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dbd3c-d0c0-416e-b6fa-6ea50ee26db7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.689407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.690189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.690537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.690880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.691692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.693704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.723522 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1192/2378] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:19:17 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:19:17.726999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.740825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885767a-26a3-42d1-a8b1-1f47ce3ce7af admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.805660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.806419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.806767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.807184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.808018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.810474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.842324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1187/2379] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:19:17 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:19:17.846487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.858884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fbecd-ce9c-4f45-b7b2-1561c030e2a2 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.864031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.864707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.864983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.865310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.866052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.867686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.879133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1193/2380] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:19:17.883445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.894691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d47a-f2f3-4815-bf85-99f430cf0d66 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.900060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.900284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.900632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.900831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.901567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.903147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca31f9-5abd-45c8-b73a-d291361efbbb admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.914177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1188/2381] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:19:17.918505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:17.929571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4191-904e-4dd4-a018-b3f3b7078a61 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:17.937296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:17.938558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.939399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.939982 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] PATH_INFO: `/v3/projects/cc3651ce859043d2a764fa47f06db374/users/9341552f54ff45d8b98b54522ba9e3f9/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:17.958514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3651ce859043d2a764fa47f06db374, user_id=9341552f54ff45d8b98b54522ba9e3f9, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:17.961345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:17.977337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1194/2382] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:19:17 2022] PUT /identity/v3/projects/cc3651ce859043d2a764fa47f06db374/users/9341552f54ff45d8b98b54522ba9e3f9/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:17.982514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351267-3b04-483e-b1a9-2c43bfb6a0b9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:17.982806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351267-3b04-483e-b1a9-2c43bfb6a0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:17.983086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351267-3b04-483e-b1a9-2c43bfb6a0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:18.014096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-5b351267-3b04-483e-b1a9-2c43bfb6a0b9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:18.015337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-5b351267-3b04-483e-b1a9-2c43bfb6a0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:18.078667 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1189/2383] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:18.088981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:18.091219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:18.116329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:18.169260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da477123-6241-47d2-93c6-39341f921d8f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:18.173507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:18.174275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:18.174624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:18.174970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:18.189062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:18.190564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:18.203884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1195/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:19:22.891595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:22.892081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:22.892183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:22.935646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] MFA Rules not processed for user `9341552f54ff45d8b98b54522ba9e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:22.936600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:22.961347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:23.032479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1190/2385] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:23.036905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.051181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493f8c3-50cb-4cbf-b0d5-df19fcf5c9a2 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.060102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.060790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.061075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.061348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.062105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.064463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.101942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1196/2386] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:19:23 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:19:23.106205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.118430 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1304-7d71-4c1d-aeb4-e768d6632536 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.222548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.223385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.223543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.223800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.224556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.226500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.249705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1191/2387] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:19:23 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:19:23.253618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.273580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e3dc-bf2d-4189-ad4f-2501d4102991 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.280074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.280862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.281357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.281815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.282742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.284746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.301061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1197/2388] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:19:23.303445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.314703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03f33-1d59-4a2c-98df-e00a9bca8d99 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.320310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.321508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.321508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.321508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] PATH_INFO: `/v3/projects/b21130dbb3a84e40b87422929d0b038c/users/b05a208c268d44658abceddd4601d63c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.343057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21130dbb3a84e40b87422929d0b038c, user_id=b05a208c268d44658abceddd4601d63c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.346006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.364528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1192/2389] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:19:23 2022] PUT /identity/v3/projects/b21130dbb3a84e40b87422929d0b038c/users/b05a208c268d44658abceddd4601d63c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:23.368725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.379883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdf17b-acc2-42e7-aff4-c2cf48d11932 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.461670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.462348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.462626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.462904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.463968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.465763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf10dc-60fd-450c-9e42-ecbd2bd79d00 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.478367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1198/2390] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:19:23.482719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:23.493925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380a0dd-a01d-4fc4-b4d3-fe3f1bde0ebf admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:23.502347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:23.503343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:23.503725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:23.504675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] PATH_INFO: `/v3/projects/b21130dbb3a84e40b87422929d0b038c/users/b05a208c268d44658abceddd4601d63c/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:23.506429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21130dbb3a84e40b87422929d0b038c, user_id=b05a208c268d44658abceddd4601d63c, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:23.509274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:23.520842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1193/2391] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:19:23 2022] PUT /identity/v3/projects/b21130dbb3a84e40b87422929d0b038c/users/b05a208c268d44658abceddd4601d63c/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:24.390733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:24.391786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:24.392286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:24.411883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:24.414277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:24.480821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1199/2392] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:24.504605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:24.506600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:24.604920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871df43-9111-4bdf-bd80-03482d40b12d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:24.611642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:24.612332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:24.612616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:24.612896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:24.623654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:24.625971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72901b3-2d10-4f70-ab83-b53e0c561a84 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:24.632678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:24.648135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1194/2393] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:19:26.623509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:26.625607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:26.727841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:26.817730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f63fa-817e-42e5-a8ad-c56bc32b6b2c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:26.824947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:26.825863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:26.826142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:26.827168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] PATH_INFO: `/v3/users/c7bdd8ec6d4947ee9e6f9e963a6f4efb` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:26.829379 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bdd8ec6d4947ee9e6f9e963a6f4efb)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:26.834236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:26.957818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bdd8ec6d4947ee9e6f9e963a6f4efb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:26.971359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bdd8ec6d4947ee9e6f9e963a6f4efb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:26.975998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bdd8ec6d4947ee9e6f9e963a6f4efb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:26.993148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bdd8ec6d4947ee9e6f9e963a6f4efb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.006893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bdd8ec6d4947ee9e6f9e963a6f4efb'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.018033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1200/2394] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:19:26 2022] DELETE /identity/v3/users/c7bdd8ec6d4947ee9e6f9e963a6f4efb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:27.023190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:27.023744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:27.023978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:27.053946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:27.055888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:27.136125 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1195/2395] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:27.159412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:27.161454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:27.226803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c285e-ce5c-4be0-bb01-8edf28ae9b6a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:27.231539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:27.231539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:27.231539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:27.231539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:27.242335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:27.244024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1ab99-e881-456b-808e-6abe93822bc2 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:27.270502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1201/2396] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:19:27.732042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:27.733764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:27.750361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe656232-1cc8-46ba-8aff-6e1b4c295582 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:27.758457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:27.759527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:27.759905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:27.760276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] PATH_INFO: `/v3/projects/53de01ad2f474a2fbd9c37598c1cee6a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:27.761629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53de01ad2f474a2fbd9c37598c1cee6a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:27.763997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:27.826754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53de01ad2f474a2fbd9c37598c1cee6a', 'request_id': 'req-95dbd704-877d-4333-b9a6-c50b61a9a145'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.838848 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53de01ad2f474a2fbd9c37598c1cee6a', 'request_id': 'req-95dbd704-877d-4333-b9a6-c50b61a9a145'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.851159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53de01ad2f474a2fbd9c37598c1cee6a', 'request_id': 'req-95dbd704-877d-4333-b9a6-c50b61a9a145'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.867277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] The token cache is being invalidate because project 53de01ad2f474a2fbd9c37598c1cee6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:19:27.867602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.868699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:19:27.871046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1196/2397] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:19:27 2022] DELETE /identity/v3/projects/53de01ad2f474a2fbd9c37598c1cee6a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:19:29.137697 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9fdd-b717-451b-b7cb-bc6ea62e0ab1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:29.138246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9fdd-b717-451b-b7cb-bc6ea62e0ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:29.138660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e9fdd-b717-451b-b7cb-bc6ea62e0ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:29.171375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0d6e9fdd-b717-451b-b7cb-bc6ea62e0ab1 None None] MFA Rules not processed for user `b05a208c268d44658abceddd4601d63c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:29.172681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0d6e9fdd-b717-451b-b7cb-bc6ea62e0ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:29.223454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:29.282744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1202/2398] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:29.299199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:19:29.301397 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:29.455294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbd704-877d-4333-b9a6-c50b61a9a145 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:19:29.461601 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:19:29.462687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:29.462945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:29.463233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:29.465922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:29.499391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:29.568083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:19:29.570899 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81d005-3e03-4fa6-a548-9624d610e7f0 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:19:29.588657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1197/2399] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:19:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:19:34.484234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:34.484758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:34.484877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:34.503956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] MFA Rules not processed for user `f705392900ef4f158b3bbc6f910fd67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:34.505201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:34.562985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1203/2400] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:39.583041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=18 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:44.555371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=35 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:19:47.485291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30f76-8c42-4957-8b77-0bf798569681 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:19:47.485904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30f76-8c42-4957-8b77-0bf798569681 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:19:47.485904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30f76-8c42-4957-8b77-0bf798569681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:19:47.510088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-eeb30f76-8c42-4957-8b77-0bf798569681 None None] MFA Rules not processed for user `9341552f54ff45d8b98b54522ba9e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:19:47.511331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-eeb30f76-8c42-4957-8b77-0bf798569681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:19:47.520524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1198/2401] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:19:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:19:57.502681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:20:00.989237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:20:00.990743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:01.048887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b22768-8a60-4bdb-9e2e-700802510e5c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:20:01.052940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:20:01.053583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:01.053858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:01.054137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:01.063939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:20:01.065371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c371-00ce-4a36-a3e9-31ca5f2e523b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:20:01.074754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1204/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:20:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:20:01.853591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:01.854152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:01.854152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:01.879288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] MFA Rules not processed for user `a14ddfab91e94f6db6d7f6596ca31916`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:20:01.880602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:01.952469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1199/2403] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:20:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:20:11.070595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=26 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:20:11.944494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=25 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:20:16.381853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:16.382404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:16.382404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:16.401426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] MFA Rules not processed for user `b3fe0e12c72c48fcb9ce277ebaa910f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:20:16.402640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:16.463318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1205/2404] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:20:26.455800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=25 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:20:34.058874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:20:34.060465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:34.128207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f0dd-f4ba-4ae9-829f-6ccdcd21715a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:20:34.132976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:20:34.133668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:34.133923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:34.134238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:34.136287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:34.152287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:20:34.153952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:20:34.165823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1200/2405] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:20:42.027395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:20:42.040863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f035f-5b57-4190-ab67-6b8fcd779a80 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:20:42.045101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:20:42.045764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:42.046080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:42.046357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:42.048441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:42.064014 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:20:42.065471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f32959-8867-4d22-a4bf-725154d51d44 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:20:42.077830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1206/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:20:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:20:44.160617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=26 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:20:48.482695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:20:48.492083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12cdd4-7d34-4e33-aac9-117e0adb9696 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:20:48.496097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:20:48.497082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:48.497082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:48.497285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:48.499205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:48.511817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:20:48.513800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:20:48.527835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1201/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:20:50.568114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:50.569114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:50.569114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:50.587416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] MFA Rules not processed for user `f705392900ef4f158b3bbc6f910fd67b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:20:50.588692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:50.595937 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1207/2408] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:20:53.464693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:20:53.481033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c2db1-c2a9-46e0-bd77-4af09356815c service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:20:53.489289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:20:53.489956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:20:53.490232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:20:53.490498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:20:53.492773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:20:53.505020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:20:53.506634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:20:53.517051 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1202/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:00.581723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=9 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:03.511969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=12 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:10.522965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:10.524506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:10.538266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3a924-d94c-4d18-ab4a-ea94dd98dd27 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:10.542560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:10.543274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:10.543584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:10.543863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:10.546264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:10.561293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:10.563718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:10.576505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1208/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:12.497951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:12.499626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:12.515303 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1615ad-286d-4b09-8f89-0633c7135fd1 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:12.521442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:12.522102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:12.522379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:12.522648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:12.524649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:12.536226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:12.537971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:12.548838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1203/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:13.983728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:13.996170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ef092-54a0-434e-9a68-241a61efd40e service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:14.000153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:14.000830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:14.001111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:14.001386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:14.003537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:14.074790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:14.076419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040dc33-5e27-4c39-88bc-99e15c637b17 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:14.088543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1209/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:15.132705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:15.145551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bf090-0a22-4a14-bad4-fc60d79bc162 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:15.150281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:15.151668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.152127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.152542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.155285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:15.170074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:15.171582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ca3de-275d-4da6-812f-1680f21bf1a6 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:15.181164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1204/2413] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:15.356604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2223-eb98-47dd-a912-aa71ea9cfe8b None None] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.357690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2223-eb98-47dd-a912-aa71ea9cfe8b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.358124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a2223-eb98-47dd-a912-aa71ea9cfe8b None None] PATH_INFO: `/` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.360077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1210/2414] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:21:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:21:15.368152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb30f2-fb8d-47b1-9eb8-8128e430ce44 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.368745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb30f2-fb8d-47b1-9eb8-8128e430ce44 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.369252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb30f2-fb8d-47b1-9eb8-8128e430ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.417016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b7eb30f2-fb8d-47b1-9eb8-8128e430ce44 None None] MFA Rules not processed for user `fcb115c5cad84f55b143c392c8c23dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:15.418355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b7eb30f2-fb8d-47b1-9eb8-8128e430ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:15.427322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1205/2415] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:15.443952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a2223-eb98-47dd-a912-aa71ea9cfe8b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:15.461368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a2223-eb98-47dd-a912-aa71ea9cfe8b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:15.478456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:15.479597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.480087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.480566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.497132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:15.500667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a609d-3172-4b8f-bf71-ad200c7c574a service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:15.515872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1211/2416] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:15.671392 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f351-2ae8-4abc-b8ae-0104b94a97f2 None None] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.672175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f351-2ae8-4abc-b8ae-0104b94a97f2 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.672742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f351-2ae8-4abc-b8ae-0104b94a97f2 None None] PATH_INFO: `/` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.676001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1206/2417] 10.222.0.18 () {54 vars in 921 bytes} [Thu Jul 21 12:21:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 12:21:15.682856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.683366 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.683985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.717311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] MFA Rules not processed for user `bccc158b75eb41ca8431bbb67af65260`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:15.719813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:15.736478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1212/2418] 10.222.0.18 () {58 vars in 1018 bytes} [Thu Jul 21 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:15.750151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f351-2ae8-4abc-b8ae-0104b94a97f2 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:15.762822 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f351-2ae8-4abc-b8ae-0104b94a97f2 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:15.766693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:15.767513 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:15.767807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:15.768086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:15.780066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:15.781669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:15.794260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1207/2419] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:21:15.875875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:15.877468 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:15.985345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49f41a-c9bb-4d65-86a0-bdb82a0ee7b5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:16.002587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:16.003775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:16.005235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:16.005749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:16.010666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:16.036825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:16.040421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:16.058788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1213/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:18.741666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:18.743269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:18.776218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce83f0-f8a3-4435-9606-1913feda1035 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:18.793976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:18.795505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:18.795806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:18.796081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] PATH_INFO: `/v3/users/a14ddfab91e94f6db6d7f6596ca31916` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:18.797447 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14ddfab91e94f6db6d7f6596ca31916)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:18.799717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:18.886725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14ddfab91e94f6db6d7f6596ca31916'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:18.900938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14ddfab91e94f6db6d7f6596ca31916'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:18.920769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14ddfab91e94f6db6d7f6596ca31916'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:18.921873 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14ddfab91e94f6db6d7f6596ca31916'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:18.935806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9a1190a4-2e36-4332-b424-48cfe08db46f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14ddfab91e94f6db6d7f6596ca31916'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:18.945882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1208/2421] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:18 2022] DELETE /identity/v3/users/a14ddfab91e94f6db6d7f6596ca31916 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:19.195673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.222737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad7a5-ba21-4d31-8544-2373cae1f142 service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.276948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:19.288597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.289487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.289768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.290049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] PATH_INFO: `/v3/projects/f3726ac8b83847468671bd930deb5492` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.291335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3726ac8b83847468671bd930deb5492)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.293705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.367910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3726ac8b83847468671bd930deb5492', 'request_id': 'req-e68a28d7-34d4-4c74-9943-51895d564c5e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:19.378727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3726ac8b83847468671bd930deb5492', 'request_id': 'req-e68a28d7-34d4-4c74-9943-51895d564c5e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:19.392352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3726ac8b83847468671bd930deb5492', 'request_id': 'req-e68a28d7-34d4-4c74-9943-51895d564c5e'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:19.414762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] The token cache is being invalidate because project f3726ac8b83847468671bd930deb5492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:19.415096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:19.416942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e68a28d7-34d4-4c74-9943-51895d564c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:19.422256 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1214/2422] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:19 2022] DELETE /identity/v3/projects/f3726ac8b83847468671bd930deb5492 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:19.424171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.424448 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.424719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.449202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:19.450806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:19.535358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1209/2423] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:19.543443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.543687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.543969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.564556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:19.565848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:19.574416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1215/2424] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:19.579391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.590202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca093a-9061-41e5-a810-a7bb9dbc2b58 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.595535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.596221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.596507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.596778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.597527 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:21:19.597835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.599469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.612908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1210/2425] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:21:19.617564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.627430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f87dc0-9d52-4693-ab36-c506d4d7bbd1 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.632609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.633295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.633576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.633853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.634560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.636565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.666738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1216/2426] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:19 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:19.671320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.686516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336ed1-b358-4a8a-8f4a-0e72889e3829 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.703762 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:19.767348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.768121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.768338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.768617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.769328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.771286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.797299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1211/2427] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:19 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:19.801544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.812424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf0d72-6923-4a46-ba3d-5b616ebbfc62 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.818060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.818723 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.819002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.819274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.820042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.821659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.832983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1217/2428] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:21:19.837014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.847516 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3962-3c3d-4199-a1b0-61e689a0d912 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.854366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.855343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.855760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.856121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.856946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.858765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aceda8-b2dc-456c-bd9e-24e0a9452925 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.872704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1212/2429] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:19.878222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:19.892478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a0e58-4d55-4223-94cc-640c5e224950 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:19.898731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:19.899686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.899970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.900246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] PATH_INFO: `/v3/projects/18e963a50a52462db1038c09c81bdcfa/users/16771f87ccfc4a94a58c22aac2f6fa8f/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.918837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e963a50a52462db1038c09c81bdcfa, user_id=16771f87ccfc4a94a58c22aac2f6fa8f, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:19.921559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:19.938316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1218/2430] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:19 2022] PUT /identity/v3/projects/18e963a50a52462db1038c09c81bdcfa/users/16771f87ccfc4a94a58c22aac2f6fa8f/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:19.950248 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:19.950698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:19.951178 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:19.986332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] MFA Rules not processed for user `16771f87ccfc4a94a58c22aac2f6fa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:19.987759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:20.063698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1213/2431] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:20.068561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.145718 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40354-6508-41d5-a754-17b896a3b015 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.154235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.154985 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.155266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.155575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.156927 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.158890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.186614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1219/2432] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:20 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:20.193588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.207313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1e87b-570c-4ddb-80ce-15907cc89865 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.278602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.279291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.279623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.279904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.280635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.282849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.305868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1214/2433] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:20 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:20.310244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.316693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:20.324968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d9ce2-8f53-44f3-9d4d-62c860c7b5e0 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.332287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.333073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.333368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.333654 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.334421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.336494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.350137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1220/2434] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:20.355026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.365930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfc223-db5b-463f-a2fb-3b26d76a4c5b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.370892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.371851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.372138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.372417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] PATH_INFO: `/v3/projects/8abe99d9ee6148fda492b61cd6e041c3/users/d0dbef4d70c248cd801d5757d9e29615/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.382205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:20.391000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abe99d9ee6148fda492b61cd6e041c3, user_id=d0dbef4d70c248cd801d5757d9e29615, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.394261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.410618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1215/2435] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:20 2022] PUT /identity/v3/projects/8abe99d9ee6148fda492b61cd6e041c3/users/d0dbef4d70c248cd801d5757d9e29615/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:20.414725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.425480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b099d-b6ae-40bc-895c-534f0cc393e1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.498288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.499308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.499704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.500174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.501592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.504526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a291fc-92b0-48a7-9d49-7aa189a42cc8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.522624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1221/2436] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:20.527366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.540478 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67a175-6ef2-46d6-8091-62abf0584343 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.546828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.547815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.548129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.548371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] PATH_INFO: `/v3/projects/8abe99d9ee6148fda492b61cd6e041c3/users/d0dbef4d70c248cd801d5757d9e29615/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.550738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abe99d9ee6148fda492b61cd6e041c3, user_id=d0dbef4d70c248cd801d5757d9e29615, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.554336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.568179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1216/2437] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:20 2022] PUT /identity/v3/projects/8abe99d9ee6148fda492b61cd6e041c3/users/d0dbef4d70c248cd801d5757d9e29615/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:20.578974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.579331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.579635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.609672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] MFA Rules not processed for user `d0dbef4d70c248cd801d5757d9e29615`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:20.611051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:20.701456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1222/2438] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:20.718083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.719930 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:20.786032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f3a89-c192-42f5-b878-118ca6955a31 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.791117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.792305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.792595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.793310 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.804025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.805501 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:20.816905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1217/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:20.913500 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:20.924307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33ab3c-a92d-4e05-8e1e-92ddde425f00 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:20.928786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:20.929479 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:20.929781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:20.930052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:20.994938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:20.996413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55934772-55d4-41ee-aba9-3c2269f1c927 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:21.006515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1223/2440] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:30.812073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:31.001842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:31.766804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:31.850523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8f23e-0dbc-4db1-bce6-aa5ff966270c service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:31.856466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:31.857542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:31.857837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:31.858116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] PATH_INFO: `/v3/users/16771f87ccfc4a94a58c22aac2f6fa8f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:31.859505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16771f87ccfc4a94a58c22aac2f6fa8f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:31.861578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:31.918854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16771f87ccfc4a94a58c22aac2f6fa8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:31.927650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16771f87ccfc4a94a58c22aac2f6fa8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:31.942150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16771f87ccfc4a94a58c22aac2f6fa8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:31.942912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16771f87ccfc4a94a58c22aac2f6fa8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:31.950669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16771f87ccfc4a94a58c22aac2f6fa8f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:31.960885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1218/2441] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:31 2022] DELETE /identity/v3/users/16771f87ccfc4a94a58c22aac2f6fa8f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:31.966279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:31.966566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:31.966846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:31.986079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:31.988006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:32.056105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1224/2442] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:32.067994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:32.069593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:32.130773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00112d-d7be-4cd4-89da-c90b6e9c2c37 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:32.134579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:32.135247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:32.136030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:32.136319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:32.146788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:32.148405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:32.161699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1219/2443] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:32.455370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:32.456990 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:32.470261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57c336-cc24-49e8-94aa-7ed8448aa148 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:32.475693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:32.476702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:32.477053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:32.477328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] PATH_INFO: `/v3/projects/18e963a50a52462db1038c09c81bdcfa` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:32.478689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e963a50a52462db1038c09c81bdcfa)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:32.480711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:32.537181 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e963a50a52462db1038c09c81bdcfa', 'request_id': 'req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.543685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e963a50a52462db1038c09c81bdcfa', 'request_id': 'req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.549398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e963a50a52462db1038c09c81bdcfa', 'request_id': 'req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.563897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] The token cache is being invalidate because project 18e963a50a52462db1038c09c81bdcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:32.564184 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.564831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.566462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1225/2444] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:32 2022] DELETE /identity/v3/projects/18e963a50a52462db1038c09c81bdcfa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:32.570620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:32.583080 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb820a5-f90c-474b-a3f8-2208bc823cf1 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:32.609109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:32.652279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:32.653185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:32.653450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:32.653722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] PATH_INFO: `/v3/users/d0dbef4d70c248cd801d5757d9e29615` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:32.654936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0dbef4d70c248cd801d5757d9e29615)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:32.656927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:32.715440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0dbef4d70c248cd801d5757d9e29615'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.726344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0dbef4d70c248cd801d5757d9e29615'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.741241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0dbef4d70c248cd801d5757d9e29615'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.741241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0dbef4d70c248cd801d5757d9e29615'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.748905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ed6b2b30-58ea-4c1f-8e2b-90806e56485a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0dbef4d70c248cd801d5757d9e29615'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:32.755358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1220/2445] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:32 2022] DELETE /identity/v3/users/d0dbef4d70c248cd801d5757d9e29615 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:33.096979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.098689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:33.117091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1684c-9505-4949-b0b2-31f1e5e5f2bd admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.184464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.185447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.185713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.186397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] PATH_INFO: `/v3/projects/8abe99d9ee6148fda492b61cd6e041c3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.187741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abe99d9ee6148fda492b61cd6e041c3)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.189797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.230977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:33.252315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abe99d9ee6148fda492b61cd6e041c3', 'request_id': 'req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:33.261018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abe99d9ee6148fda492b61cd6e041c3', 'request_id': 'req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:33.271317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abe99d9ee6148fda492b61cd6e041c3', 'request_id': 'req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:33.288489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] The token cache is being invalidate because project 8abe99d9ee6148fda492b61cd6e041c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:33.288799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:33.289461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e1ccef17-798d-42f2-b50f-ac415a2ec7fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:33.291117 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1226/2446] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:33 2022] DELETE /identity/v3/projects/8abe99d9ee6148fda492b61cd6e041c3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:33.297254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.297813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.297813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.318102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:33.319699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:33.403939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1221/2447] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:33.413198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.413589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.413912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.436370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:33.437872 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:33.446289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1227/2448] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:33.451019 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.463775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb6af-6c68-4dbc-820f-92b96041d18e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.469790 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.470506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.470780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.471060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.471852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:21:33.472212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.474626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.491953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1222/2449] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:21:33.497124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.512778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b93d7-bb85-482d-b011-22b645c2a67d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.520509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.521319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.521462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.521733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.522428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.525233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.555669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1228/2450] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:33.560456 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.573398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c01d-71d3-49a7-a556-35e34e9bf2b7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.619451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:33.672802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.673979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.674490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.674990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.676339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.680257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.710537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1223/2451] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:33.716081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.728197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7ed3-aa6c-4560-abce-f4539d391e20 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.733290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.733989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.734275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.734561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.735279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.736960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.752959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1229/2452] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:33.758572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.770156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf76d2e-50c1-4537-8a71-7980bdad3e7e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.776004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.776823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.777171 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.777445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.778307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.779993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad68e96-d144-4d3d-a8aa-cdf269fc6140 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.791041 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1224/2453] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:33.795528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:33.805831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0498-f853-4db9-a3a0-b48fef636bf0 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:33.810444 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:33.811738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.812124 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.812576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] PATH_INFO: `/v3/projects/4f126339108f4ce1ae85142445140a72/users/321a1b85c86b4c01a499c03085882b12/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.827914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f126339108f4ce1ae85142445140a72, user_id=321a1b85c86b4c01a499c03085882b12, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:33.830606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:33.846244 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1230/2454] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:33 2022] PUT /identity/v3/projects/4f126339108f4ce1ae85142445140a72/users/321a1b85c86b4c01a499c03085882b12/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:33.859853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:33.860143 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:33.860425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:33.889881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] MFA Rules not processed for user `321a1b85c86b4c01a499c03085882b12`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:33.891188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:33.972221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1225/2455] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:33.976849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.043441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676513a-9fcf-4d51-a763-799d9910eb8d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.049012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.049701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.050001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.050421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.051162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.053151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.084386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1231/2456] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:33 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:34.089689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.102780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340c2a6-3c5c-46db-aabb-04058646cc3c None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.167003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.167852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.168017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.168191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.170061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.170853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.194010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1226/2457] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:34 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:21:34.198161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.209281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e720a20-f70a-4b46-bcbd-13cc2093d8e4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.213925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.214592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.214869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.215140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.215924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.217555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.228666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1232/2458] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:34.233066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.244989 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324662f-9814-40a9-9dc8-1c83e9ebaaf0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.249836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.250728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.251006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.251277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] PATH_INFO: `/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c/users/3771555595a74a03a0e8efa3eaa1d6b8/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.268300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba9ffc99f5d4370a558e874bd8fc99c, user_id=3771555595a74a03a0e8efa3eaa1d6b8, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.270993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.285836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1227/2459] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:34 2022] PUT /identity/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c/users/3771555595a74a03a0e8efa3eaa1d6b8/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:34.291235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.302241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8e464-3ab0-4b33-ab6b-858077425ba4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.326946 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:34.370374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.371056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.371353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.371637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.372397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.374022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764186f-de46-419d-a8ee-eee6af529555 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.384794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1233/2460] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:34.389226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.395977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:34.404586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381792a-5416-4671-8b38-15338d7606b2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.410684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.411624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.411912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.412193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] PATH_INFO: `/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c/users/3771555595a74a03a0e8efa3eaa1d6b8/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.414871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba9ffc99f5d4370a558e874bd8fc99c, user_id=3771555595a74a03a0e8efa3eaa1d6b8, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.417627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.429723 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1228/2461] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:34 2022] PUT /identity/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c/users/3771555595a74a03a0e8efa3eaa1d6b8/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:34.440794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.441081 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.441359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.473309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] MFA Rules not processed for user `3771555595a74a03a0e8efa3eaa1d6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:34.474571 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:34.550098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1234/2462] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:34.563677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.565453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:34.628440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782ffac-211e-4a8f-8de4-8e4e88edc753 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.632444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.633108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.633403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.633675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.644620 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.646105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dc9e3-203e-4c60-b3d4-e14e9c2aae5a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.656973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1229/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:34.713435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:34.714977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:34.728300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83460044-0c94-4b35-a481-80d838c14a3b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:34.732635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:34.733303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:34.733583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:34.733932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:34.808269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:34.810821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:34.827243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1235/2464] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:35.894433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:35.894973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:35.894973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:35.914060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:35.915279 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:35.973184 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1230/2465] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:21:36.030499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:36.032036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:36.087107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:36.113778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e33ad7-b2be-454c-b784-cc2649d4646c service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:36.119130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:36.120000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:36.120293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:36.120610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:36.130152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:36.131805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:36.141048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1236/2466] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:21:37.452339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:37.455739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:37.609994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b20cc-1bdc-4b52-a2a6-ed3b1a7abf79 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:37.618373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:37.619124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:37.619353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:37.619692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:37.622464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:37.637924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:37.639726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb6d2a-70fa-44f2-bcfb-dd6f4c7dfbcb service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:37.653094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1231/2467] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:38.640988 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:38.713481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26d6c-3677-4c9f-8778-18b1e1ad37db service swift] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:38.718789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:38.719740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:38.720021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:38.720298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] PATH_INFO: `/v3/users/321a1b85c86b4c01a499c03085882b12` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:38.721645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=321a1b85c86b4c01a499c03085882b12)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:38.723732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:38.781258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321a1b85c86b4c01a499c03085882b12'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:38.788442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321a1b85c86b4c01a499c03085882b12'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:38.789135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321a1b85c86b4c01a499c03085882b12'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:38.802299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321a1b85c86b4c01a499c03085882b12'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:38.813373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321a1b85c86b4c01a499c03085882b12'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:38.821147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1237/2468] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:38 2022] DELETE /identity/v3/users/321a1b85c86b4c01a499c03085882b12 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:38.826424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:38.827245 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:38.827766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:38.849104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:38.850557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:38.875792 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:38.935215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1232/2469] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:38.946845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:38.948613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:39.021578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fdbbb-51a0-446f-89dc-cf12d6780811 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:39.029120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:39.030008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:39.030270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:39.030563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:39.046363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:39.051415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:39.054318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:39.073121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1238/2470] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:39.474268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:39.475823 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:39.496236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2f2a-4bf0-42c6-8d7f-3b9d6a78d96e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:39.505948 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:39.508016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:39.508591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:39.508955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] PATH_INFO: `/v3/projects/4f126339108f4ce1ae85142445140a72` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:39.510407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f126339108f4ce1ae85142445140a72)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:39.512429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:39.616886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f126339108f4ce1ae85142445140a72', 'request_id': 'req-b4695125-fe06-43f4-8ab6-85c708d4469b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.628492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f126339108f4ce1ae85142445140a72', 'request_id': 'req-b4695125-fe06-43f4-8ab6-85c708d4469b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.643857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f126339108f4ce1ae85142445140a72', 'request_id': 'req-b4695125-fe06-43f4-8ab6-85c708d4469b'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.665682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] The token cache is being invalidate because project 4f126339108f4ce1ae85142445140a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:39.666739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.668398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.676209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1233/2471] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:39 2022] DELETE /identity/v3/projects/4f126339108f4ce1ae85142445140a72 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:39.681026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:39.700080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85161900-ef7d-4be5-a643-e073460aa634 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:39.792394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:39.794213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:39.794800 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:39.795374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] PATH_INFO: `/v3/users/3771555595a74a03a0e8efa3eaa1d6b8` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:39.798198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3771555595a74a03a0e8efa3eaa1d6b8)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:39.802261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:39.888769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3771555595a74a03a0e8efa3eaa1d6b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.902107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3771555595a74a03a0e8efa3eaa1d6b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.903289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3771555595a74a03a0e8efa3eaa1d6b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.925168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3771555595a74a03a0e8efa3eaa1d6b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.935520 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7ba08c9a-bf8e-4d0c-b505-a7a08fe2c848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3771555595a74a03a0e8efa3eaa1d6b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:39.944303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1239/2472] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:39 2022] DELETE /identity/v3/users/3771555595a74a03a0e8efa3eaa1d6b8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:40.384791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:40.386373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:40.402348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4695125-fe06-43f4-8ab6-85c708d4469b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:40.520946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:40.521988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:40.522268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:40.522636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] PATH_INFO: `/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:40.525939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba9ffc99f5d4370a558e874bd8fc99c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:40.527097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:40.612544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:40.614121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba9ffc99f5d4370a558e874bd8fc99c', 'request_id': 'req-0dde92ee-6ec6-4df4-8663-e1e13185882a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:40.627856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba9ffc99f5d4370a558e874bd8fc99c', 'request_id': 'req-0dde92ee-6ec6-4df4-8663-e1e13185882a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:40.639409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba9ffc99f5d4370a558e874bd8fc99c', 'request_id': 'req-0dde92ee-6ec6-4df4-8663-e1e13185882a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:40.670990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] The token cache is being invalidate because project 7ba9ffc99f5d4370a558e874bd8fc99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:40.671331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:40.675538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0dde92ee-6ec6-4df4-8663-e1e13185882a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:40.678975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1234/2473] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:40 2022] DELETE /identity/v3/projects/7ba9ffc99f5d4370a558e874bd8fc99c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:40.684145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:40.685503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:40.686429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:40.706361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:40.708215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:40.808277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1240/2474] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:40.831635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:40.831871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:40.832137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:40.859023 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:40.860401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:40.871636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1235/2475] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:40.875548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:40.888059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf210ec-82aa-44de-aa23-af9975997242 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:40.893589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:40.894255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:40.894548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:40.894823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:40.895632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:21:40.895951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:40.897659 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:40.917737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1241/2476] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:21:40.923126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:40.937095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24fa33-c69a-4cbb-b6e7-5280332cdb60 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:40.942285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:40.943460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:40.943971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:40.944602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:40.945771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:40.949330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:40.992665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1236/2477] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:40 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:40.997168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.003290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:41.010390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59180cc0-3834-47ac-a352-e2294a23cea9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.100986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.101843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.102216 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.102552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.103363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.105652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.136519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1242/2478] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:40 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:41.138868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.162360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d332eac-68d9-4480-aaf8-331117eff629 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.171308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.173353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.174147 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.174861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.176582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.180341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.197328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1237/2479] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:41.200348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.213766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434dc30-6b86-432d-96dd-82bb50f7c8cf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.219266 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.220150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.220420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.220696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.221487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.223140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab23f-aeed-4ce3-af86-68ce06280af6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.235782 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1243/2480] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:21:41.240727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.259540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f84a-737a-4316-b3b7-7b348ca7ff96 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.267504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.268846 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.269340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.269696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] PATH_INFO: `/v3/projects/97a6299ccee8430cba0af3a13a05c417/users/10e109d13ae14ff18aee10e47b50c206/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.302368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a6299ccee8430cba0af3a13a05c417, user_id=10e109d13ae14ff18aee10e47b50c206, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.305206 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.335361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1238/2481] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:41 2022] PUT /identity/v3/projects/97a6299ccee8430cba0af3a13a05c417/users/10e109d13ae14ff18aee10e47b50c206/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:41.338018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.342546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.342546 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.378851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] MFA Rules not processed for user `10e109d13ae14ff18aee10e47b50c206`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:41.378851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:41.521795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1244/2482] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:41.526945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.626384 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670308-3012-4487-be63-5d0af0e421bf admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.641874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.645012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.647462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.648044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.649433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.654079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.705175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1239/2483] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:41 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:41.709261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.727415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3566c6-9a2b-4e50-bb5e-656898096253 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.819893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.820672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.821020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.821361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.822176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.824376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.866854 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:41.879457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1245/2484] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:41 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:41.879754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.893672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0bee4-dbd9-4d15-ac6b-0cc837f281df admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.899207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.900021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.900373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.900711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:41.901533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:41.903375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:41.935016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1240/2485] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:41.939100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:41.962876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9b0de-b2b5-4dcc-a56c-82fefbfd2116 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:41.973325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:41.974437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:41.974579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:41.974795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] PATH_INFO: `/v3/projects/417a7eeda078489ead88409e88f1350d/users/fcc55f582f714a41bd7ccc422a0abd0f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.002793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417a7eeda078489ead88409e88f1350d, user_id=fcc55f582f714a41bd7ccc422a0abd0f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:42.005794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:42.026776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1246/2486] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:41 2022] PUT /identity/v3/projects/417a7eeda078489ead88409e88f1350d/users/fcc55f582f714a41bd7ccc422a0abd0f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:42.031838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:42.047009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69891f60-575d-47a4-b6db-b182cab59f9f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:42.123389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:42.144659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:42.145955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:42.147172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:42.147769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.148646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:42.151006 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee7d9f-9600-48e2-aab0-904317364059 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:42.180586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1241/2487] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:21:42.183738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:42.200033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ac1d5-2309-42b1-a3d3-e3dfe5a216a9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:42.205170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:42.206144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:42.206419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:42.206883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] PATH_INFO: `/v3/projects/417a7eeda078489ead88409e88f1350d/users/fcc55f582f714a41bd7ccc422a0abd0f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.209616 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417a7eeda078489ead88409e88f1350d, user_id=fcc55f582f714a41bd7ccc422a0abd0f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:42.215340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:42.232123 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1247/2488] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:42 2022] PUT /identity/v3/projects/417a7eeda078489ead88409e88f1350d/users/fcc55f582f714a41bd7ccc422a0abd0f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:42.255099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b782a5-6777-452a-850f-847a51ed892f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:42.256235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b782a5-6777-452a-850f-847a51ed892f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:42.256921 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b782a5-6777-452a-850f-847a51ed892f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.313133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-25b782a5-6777-452a-850f-847a51ed892f None None] MFA Rules not processed for user `fcc55f582f714a41bd7ccc422a0abd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:42.316252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-25b782a5-6777-452a-850f-847a51ed892f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:42.500287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1242/2489] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:21:42.520590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:42.523965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:42.635351 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6375a15-243a-4b65-9a85-3f5d36af8ba6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:42.641402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:42.642206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:42.642592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:42.642958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.660507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:42.663849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5312155-fdc9-4a67-9a3a-788843530db1 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:42.700215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1248/2490] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:42.719111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b782a5-6777-452a-850f-847a51ed892f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:42.825446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b782a5-6777-452a-850f-847a51ed892f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:42.837250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:42.839509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:42.840001 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:42.840481 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] PATH_INFO: `/v3/users/10e109d13ae14ff18aee10e47b50c206` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:42.844035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e109d13ae14ff18aee10e47b50c206)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:42.850526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:42.983436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e109d13ae14ff18aee10e47b50c206'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:42.997522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e109d13ae14ff18aee10e47b50c206'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:43.029815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:43.032784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e109d13ae14ff18aee10e47b50c206'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:43.034227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e109d13ae14ff18aee10e47b50c206'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:43.053637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e109d13ae14ff18aee10e47b50c206'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:43.066993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1243/2491] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:42 2022] DELETE /identity/v3/users/10e109d13ae14ff18aee10e47b50c206 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:43.073912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:43.074755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:43.074900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:43.134714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:43.137127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:43.310426 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1249/2492] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:43.326129 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:43.329180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:43.513324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82840e2-b108-447d-9e09-319b5d8e0688 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:43.517677 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:43.518347 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:43.518656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:43.518939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:43.529860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:43.531542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:43.546452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1244/2493] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:44.014409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:44.016469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:44.040703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fcda0-efd1-4a22-845c-6667a94a3fc6 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:44.064880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:44.068938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:44.069563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:44.079043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] PATH_INFO: `/v3/projects/97a6299ccee8430cba0af3a13a05c417` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:44.079043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a6299ccee8430cba0af3a13a05c417)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:44.079043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:44.178458 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a6299ccee8430cba0af3a13a05c417', 'request_id': 'req-5aab7e06-b47b-4f50-bb9e-b000e548bf90'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.192198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a6299ccee8430cba0af3a13a05c417', 'request_id': 'req-5aab7e06-b47b-4f50-bb9e-b000e548bf90'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.203344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:44.205569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a6299ccee8430cba0af3a13a05c417', 'request_id': 'req-5aab7e06-b47b-4f50-bb9e-b000e548bf90'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.232006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] The token cache is being invalidate because project 97a6299ccee8430cba0af3a13a05c417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:44.232959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.235955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.239178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1250/2494] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:44 2022] DELETE /identity/v3/projects/97a6299ccee8430cba0af3a13a05c417 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:44.244663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:44.255844 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227e0c-566c-4d1e-92f4-bf96447395b9 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:44.356494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:44.359573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:44.359573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:44.359573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] PATH_INFO: `/v3/users/fcc55f582f714a41bd7ccc422a0abd0f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:44.362503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc55f582f714a41bd7ccc422a0abd0f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:44.366100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:44.469685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc55f582f714a41bd7ccc422a0abd0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.485784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc55f582f714a41bd7ccc422a0abd0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.503868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc55f582f714a41bd7ccc422a0abd0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.504550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc55f582f714a41bd7ccc422a0abd0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.522866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-30db608c-a9c8-4cf6-aafc-35d24d9b9ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc55f582f714a41bd7ccc422a0abd0f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:44.537819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1245/2495] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:44 2022] DELETE /identity/v3/users/fcc55f582f714a41bd7ccc422a0abd0f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:45.187224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:45.189092 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:45.214046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab7e06-b47b-4f50-bb9e-b000e548bf90 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:45.357337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:45.358241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:45.358787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:45.358992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] PATH_INFO: `/v3/projects/417a7eeda078489ead88409e88f1350d` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:45.360465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=417a7eeda078489ead88409e88f1350d)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:45.362230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:45.427758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417a7eeda078489ead88409e88f1350d', 'request_id': 'req-118468ac-0d29-4ae6-a31e-27928a5ed159'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:45.436201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417a7eeda078489ead88409e88f1350d', 'request_id': 'req-118468ac-0d29-4ae6-a31e-27928a5ed159'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:45.448199 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417a7eeda078489ead88409e88f1350d', 'request_id': 'req-118468ac-0d29-4ae6-a31e-27928a5ed159'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:45.468340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] The token cache is being invalidate because project 417a7eeda078489ead88409e88f1350d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:45.468714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:45.469846 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-118468ac-0d29-4ae6-a31e-27928a5ed159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:45.472667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1251/2496] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:45 2022] DELETE /identity/v3/projects/417a7eeda078489ead88409e88f1350d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:45.480055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:45.480681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:45.480951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:45.511376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:45.513405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:45.520787 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:45.642682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1246/2497] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:45.650964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:45.651304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:45.651577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:45.673499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:45.675110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:45.693836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1252/2498] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:21:45.700843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:45.718098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7681e96-5361-4226-ac26-cf0a59437919 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:45.727055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:45.728234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:45.732180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:45.732180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:45.732180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:21:45.732180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:45.734661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:45.757259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1247/2499] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:21:45.763205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:45.779896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4d92-658b-4a2e-a297-18202a8ffef6 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:45.789110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:45.789984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:45.790398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:45.790776 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:45.791709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:45.796833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:45.847535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1253/2500] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:45 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:21:45.852785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:45.890895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ec288-6f72-4ab7-99a5-2edc48e92887 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.066985 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.068268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.068749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.071666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.073208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.075744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.124643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1248/2501] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:45 2022] POST /identity/v3/users => generated 284 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:21:46.136068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.158822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57428f7-35da-4c51-aa57-cd45ae85b71e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.168675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.171232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.171877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.172562 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.174211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.177888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.200719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1254/2502] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:46.207159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.223459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5fc0c-6dbe-49e6-8410-1c4c228fa6ed admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.230301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.231097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.231624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.231935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.233084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.236083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7abec-d88c-4f88-b6c2-9fb0f6674578 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.252886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1249/2503] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:46.262969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.275382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d95e-7a81-4307-98d9-794e6ed8259a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.280905 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.282080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.282421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.284067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] PATH_INFO: `/v3/projects/4811b046966d4b46a3677ad05d822a83/users/2edc05a54ebd4c9abfe4c66bd4cdb641/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.324405 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4811b046966d4b46a3677ad05d822a83, user_id=2edc05a54ebd4c9abfe4c66bd4cdb641, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.329308 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.353679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1255/2504] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:46 2022] PUT /identity/v3/projects/4811b046966d4b46a3677ad05d822a83/users/2edc05a54ebd4c9abfe4c66bd4cdb641/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:46.370539 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.370922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.371308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.403882 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] MFA Rules not processed for user `2edc05a54ebd4c9abfe4c66bd4cdb641`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:46.405239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:46.482117 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1250/2505] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:46.487862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.520352 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:46.604112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d4e1-38e8-4a38-b6e8-e636de3322c3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.610463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.611151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.611436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.611747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.612460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.614430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.644498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1256/2506] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:46 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:46.648841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.657018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:46.665156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099248-a9a3-4a8e-a0a4-5d29c5b3b01e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.772156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.773324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.773797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.774275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.775729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.779748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.808821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1251/2507] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:46 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:46.815726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.837241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b2aba-9150-47b9-9dd7-3402cf137175 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.843837 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.844752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.845154 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.845545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.846449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.848430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.862850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1257/2508] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:46.868538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.886497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14670d0-0b70-4923-8785-efae768f7f67 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:46.895836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:46.897466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:46.897962 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:46.898464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] PATH_INFO: `/v3/projects/7742b3dce8684d1b884054513e80e3de/users/3f0e5ef20c0e4e05a37fa718c956106e/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:46.928756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7742b3dce8684d1b884054513e80e3de, user_id=3f0e5ef20c0e4e05a37fa718c956106e, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:46.931993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:46.952767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1252/2509] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:46 2022] PUT /identity/v3/projects/7742b3dce8684d1b884054513e80e3de/users/3f0e5ef20c0e4e05a37fa718c956106e/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:46.958676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:46.970923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ec2a-c45f-4ff9-a3f8-f7711c3f4f83 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:47.039333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:47.040064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:47.040342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:47.040617 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:47.041365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:47.043065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072367b-417f-4b92-baba-22a7921b65f4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:47.057054 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1258/2510] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:21:47.064040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:47.082607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b278e-9b57-43ac-9178-b3ed06ef68c0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:47.092578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:47.095253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:47.095746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:47.096221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] PATH_INFO: `/v3/projects/7742b3dce8684d1b884054513e80e3de/users/3f0e5ef20c0e4e05a37fa718c956106e/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:47.100018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7742b3dce8684d1b884054513e80e3de, user_id=3f0e5ef20c0e4e05a37fa718c956106e, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:47.105321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:47.125497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1253/2511] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:47 2022] PUT /identity/v3/projects/7742b3dce8684d1b884054513e80e3de/users/3f0e5ef20c0e4e05a37fa718c956106e/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:47.147340 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:47.147802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:47.148127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:47.182069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] MFA Rules not processed for user `3f0e5ef20c0e4e05a37fa718c956106e`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:47.183320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:47.278836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1259/2512] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:47.297668 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:47.300562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:47.391071 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650cc97-9d18-47ff-8425-9dfca1f78fd6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:47.395664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:47.396360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:47.396701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:47.397046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:47.409707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:47.411894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:47.438747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1254/2513] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:47.538326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:47.558447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc83fe-fb24-4f23-8962-efb58acabd7b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:47.569161 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:47.570406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:47.570908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:47.571564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:47.591790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:47.668235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:47.669796 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736fbb-7db6-44cc-b2fb-1276f26e580d service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:47.683330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1260/2514] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:53.241150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:53.356707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ce100-073a-46d6-add8-e4fada69038f service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:53.362813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:53.363974 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:53.364322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:53.364556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] PATH_INFO: `/v3/users/2edc05a54ebd4c9abfe4c66bd4cdb641` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:53.365957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2edc05a54ebd4c9abfe4c66bd4cdb641)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:53.368045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:53.370103 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:53.441355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2edc05a54ebd4c9abfe4c66bd4cdb641'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:53.450159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2edc05a54ebd4c9abfe4c66bd4cdb641'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:53.469739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2edc05a54ebd4c9abfe4c66bd4cdb641'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:53.472102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2edc05a54ebd4c9abfe4c66bd4cdb641'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:53.484127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2edc05a54ebd4c9abfe4c66bd4cdb641'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:53.493552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1255/2515] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:53 2022] DELETE /identity/v3/users/2edc05a54ebd4c9abfe4c66bd4cdb641 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:53.498822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:53.499815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:53.499815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:53.522118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:53.524287 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:53.622289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1261/2516] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:21:53.635991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:53.637492 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:53.716135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e86fd-95b5-42f3-bfa9-e6594da4a5bc admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:53.720215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:53.720947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:53.721235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:53.721496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:53.732215 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:53.733810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:53.745327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1256/2517] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:21:54.238827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:54.241038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:54.254208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0353d-41dc-4d15-9194-501d22523d40 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:54.261329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:54.262368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:54.262733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:54.263062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] PATH_INFO: `/v3/projects/4811b046966d4b46a3677ad05d822a83` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:54.264593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4811b046966d4b46a3677ad05d822a83)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:54.267168 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:54.354611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4811b046966d4b46a3677ad05d822a83', 'request_id': 'req-4bf2711a-f7d1-42af-9cff-56bf960da868'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.367084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4811b046966d4b46a3677ad05d822a83', 'request_id': 'req-4bf2711a-f7d1-42af-9cff-56bf960da868'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.378849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4811b046966d4b46a3677ad05d822a83', 'request_id': 'req-4bf2711a-f7d1-42af-9cff-56bf960da868'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.402246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] The token cache is being invalidate because project 4811b046966d4b46a3677ad05d822a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:54.402803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.403844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.406024 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1262/2518] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:54 2022] DELETE /identity/v3/projects/4811b046966d4b46a3677ad05d822a83 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:54.410525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:54.422259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df81d-4e7c-410f-b3df-327acf655858 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:54.490704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:54.491808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:54.492086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:54.492361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] PATH_INFO: `/v3/users/3f0e5ef20c0e4e05a37fa718c956106e` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:54.493643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0e5ef20c0e4e05a37fa718c956106e)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:54.495653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:54.522821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:54.565063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0e5ef20c0e4e05a37fa718c956106e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.577491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0e5ef20c0e4e05a37fa718c956106e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.597471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0e5ef20c0e4e05a37fa718c956106e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.598673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0e5ef20c0e4e05a37fa718c956106e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.612738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-13c6ac83-268b-4415-9705-6b03ab22457b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0e5ef20c0e4e05a37fa718c956106e'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:54.625142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1257/2519] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:21:54 2022] DELETE /identity/v3/users/3f0e5ef20c0e4e05a37fa718c956106e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:55.004672 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.006355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:55.020509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2711a-f7d1-42af-9cff-56bf960da868 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.059932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:55.100056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.101480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.101652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.101953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] PATH_INFO: `/v3/projects/7742b3dce8684d1b884054513e80e3de` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.104696 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7742b3dce8684d1b884054513e80e3de)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.106735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.109991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.109991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.110745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.138911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:55.141327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:55.182015 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7742b3dce8684d1b884054513e80e3de', 'request_id': 'req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:55.192613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7742b3dce8684d1b884054513e80e3de', 'request_id': 'req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:55.201552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7742b3dce8684d1b884054513e80e3de', 'request_id': 'req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:55.218404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] The token cache is being invalidate because project 7742b3dce8684d1b884054513e80e3de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:21:55.219296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:55.221341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f41d6955-430f-49cf-92ab-ff28eeb3b9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:21:55.223957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1263/2520] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:21:55 2022] DELETE /identity/v3/projects/7742b3dce8684d1b884054513e80e3de => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:21:55.229072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71295a7-9934-4ebb-872f-6e3c6aaacaec None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.229608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71295a7-9934-4ebb-872f-6e3c6aaacaec None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.229987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71295a7-9934-4ebb-872f-6e3c6aaacaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.249462 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1258/2521] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:21:55.253252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d71295a7-9934-4ebb-872f-6e3c6aaacaec None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:55.254573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d71295a7-9934-4ebb-872f-6e3c6aaacaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:55.270461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.272417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:55.329220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1264/2522] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:55.337930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.338223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.338518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.357478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:55.359363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:55.368592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1265/2523] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:55.369092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c0441-f40e-4670-babb-e5fa633cb305 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.373492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.383566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959ef86-d709-467d-8a72-379ee77bb976 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.388788 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.389498 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.389833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.390110 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.390852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:21:55.391162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.392848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.408763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1266/2524] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:21:55.410751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.421351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:55.422393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e6fb9-633a-4394-a2df-dace44c4bfb6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.428137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.428829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.429330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.429461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.430098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.432099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.442218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.442902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.443185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.443625 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.463065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1267/2525] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:55.467424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.496544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:21:55.521043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.522700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.532102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1259/2526] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:21:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:21:55.545345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416326-7c9f-4774-a607-f051c59e8ffd admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.550407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.550991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.552386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.552386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.552386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.557261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.584215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1268/2527] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:21:55.588732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.609820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a215d9-189f-4141-afda-22d39ea4c84f service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.618173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.619739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.620352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.620721 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.622162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.625162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.664186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1260/2528] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:55.671840 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.698198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d210d-7336-4545-b4d1-5d5e212ba2b7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.707017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.708179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.708675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.709136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.710445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.713742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c423-0685-4b83-b7e4-307c4446535e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.743682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1269/2529] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:21:55.745018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:55.757995 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c072ff-002c-4bfc-ac4f-24a7f4b2b347 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:55.764913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:55.766031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.766290 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.766898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] PATH_INFO: `/v3/projects/bf930b74cefd4a8290c701304caa4ce1/users/0f6e183163294300aa297f5da4c74073/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.790611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf930b74cefd4a8290c701304caa4ce1, user_id=0f6e183163294300aa297f5da4c74073, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:55.794590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:55.819358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1261/2530] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:21:55 2022] PUT /identity/v3/projects/bf930b74cefd4a8290c701304caa4ce1/users/0f6e183163294300aa297f5da4c74073/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:21:55.838248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:55.839095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:55.839907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:55.893670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] MFA Rules not processed for user `0f6e183163294300aa297f5da4c74073`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:21:55.896306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:56.039332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1270/2531] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:21:56.057098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:21:56.059082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:21:56.186244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026a91-5b79-4c42-aa8e-c192a7b8b3a9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:21:56.215070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:21:56.218952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:21:56.219626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:21:56.221419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:21:56.249383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:21:56.251733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e63e6-a19a-4fcd-b770-d202bb7a83ff service glance] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:21:56.273700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1262/2532] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:21:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:06.031245 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:06.263176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:12.416877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:12.419517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:12.541825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8526921-9ffe-4bac-85f0-16a229ff331d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:12.548226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:12.548910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:12.549188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:12.549525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:12.554487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:12.570230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:12.571955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:12.583700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1271/2533] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:14.754465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:14.755334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:14.755912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:14.785561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:14.787681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:14.892520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1263/2534] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:14.907466 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:14.999283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09155-128c-4e73-8ee9-51d838930d06 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:15.004660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:15.005638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:15.006091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:15.006372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:15.016046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:15.021231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394d888-a982-4080-a053-b96a89ef674e service swift] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:15.031139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1272/2535] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:22:15.505005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:15.631247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54b33-a60a-4fa2-90c7-34bc6c27fda5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:15.640557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0960-5391-4536-bdac-3795826290a2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:15.641114 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0960-5391-4536-bdac-3795826290a2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:15.641608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c0960-5391-4536-bdac-3795826290a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:15.642529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:15.643573 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:15.644233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:15.644858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] PATH_INFO: `/v3/users/0f6e183163294300aa297f5da4c74073` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:15.646239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6e183163294300aa297f5da4c74073)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:15.648383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:15.669724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-ab1c0960-5391-4536-bdac-3795826290a2 None None] MFA Rules not processed for user `9341552f54ff45d8b98b54522ba9e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:15.671000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ab1c0960-5391-4536-bdac-3795826290a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:15.728452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6e183163294300aa297f5da4c74073'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:15.759348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6e183163294300aa297f5da4c74073'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:15.799743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1273/2536] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:15.804150 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6e183163294300aa297f5da4c74073'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:15.804874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6e183163294300aa297f5da4c74073'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:15.831748 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6e183163294300aa297f5da4c74073'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:15.850477 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1264/2537] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:22:15 2022] DELETE /identity/v3/users/0f6e183163294300aa297f5da4c74073 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:15.866655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:15.867452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:15.868071 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:15.909908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:15.912874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:15.962296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:16.092406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1274/2538] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:16.108644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:16.111794 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:16.198381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:16.242411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f274e8e-cdea-4d80-8e57-8824ef98e07f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:16.247570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:16.248252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:16.248683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:16.248950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:16.266996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:16.269418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba6b44-dcd6-45ee-8373-9f50e639ca02 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:16.282194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1265/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:16.787630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:16.789121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:16.803136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c93d-deb2-4fe8-9c81-79efec9d5921 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:16.808625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:16.809514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:16.809960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:16.810605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] PATH_INFO: `/v3/projects/bf930b74cefd4a8290c701304caa4ce1` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:16.811979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf930b74cefd4a8290c701304caa4ce1)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:16.813964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:16.877064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf930b74cefd4a8290c701304caa4ce1', 'request_id': 'req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:16.885485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf930b74cefd4a8290c701304caa4ce1', 'request_id': 'req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:16.892733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf930b74cefd4a8290c701304caa4ce1', 'request_id': 'req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:16.914107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] The token cache is being invalidate because project bf930b74cefd4a8290c701304caa4ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:22:16.915384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:16.915626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-1d88927c-ca9e-4eb2-ab8b-69d5bc0235db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:16.917594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1275/2540] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:22:16 2022] DELETE /identity/v3/projects/bf930b74cefd4a8290c701304caa4ce1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:16.924327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:16.925374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:16.925841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:16.950858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:16.953872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:17.075510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1266/2541] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:22:17.133029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.133380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.133623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.155189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:17.155803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:17.164801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1276/2542] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:17.171393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.182159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a2072-b1c3-4352-87db-6b908f15b20e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.188175 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.188912 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.189194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.189472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.190267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:22:17.190576 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.192307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.211192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1267/2543] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:22:17.212764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.231739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb6ac5-6956-4ccd-b908-7958905f76c7 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.240644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.241979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.242650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.243359 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.244877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.249607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.306063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1277/2544] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:22:17 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:22:17.311122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.330003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51702c7-d991-4dfe-9c2e-bf660590bae1 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.455920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.456905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.457251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.457717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.458504 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.461358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.541525 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1268/2545] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:22:17 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:22:17.541991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.558134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe277f-a3de-4876-9fca-3fb2f3f0d6f2 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.577467 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.578730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.579536 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.579899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.580980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.584786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.608832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.616178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1278/2546] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:17.621894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c1df7-e083-4335-a8f8-511042a333b5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.641226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.641951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.642277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.642560 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.643326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.650859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83810b6-3657-44ee-b585-9f23f156316c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.668194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1269/2547] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:17.670777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.686632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7f893-1018-456a-82b5-775fa2c2130f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.692818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.694169 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.694930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.695378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] PATH_INFO: `/v3/projects/53374ad6bfce4d7a9d51448ad1d4f510/users/cd30eb171a8040b4be937d568f0a4be2/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.716680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53374ad6bfce4d7a9d51448ad1d4f510, user_id=cd30eb171a8040b4be937d568f0a4be2, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.720606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:17.744428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1279/2548] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:17 2022] PUT /identity/v3/projects/53374ad6bfce4d7a9d51448ad1d4f510/users/cd30eb171a8040b4be937d568f0a4be2/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:17.747427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8582e-a9e8-4cc7-a686-8cb8a83fd1e3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.748003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8582e-a9e8-4cc7-a686-8cb8a83fd1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.748805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8582e-a9e8-4cc7-a686-8cb8a83fd1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.772078 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-84f8582e-a9e8-4cc7-a686-8cb8a83fd1e3 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:17.773594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-84f8582e-a9e8-4cc7-a686-8cb8a83fd1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:17.822287 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:17.872010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1270/2549] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:17.887449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:17.890722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:17.949515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:17.974593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c14f8-a74f-44ae-90b3-50cd473c6315 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:17.978406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:17.979179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:17.979465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:17.979774 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:17.995089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:17.997918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:18.019010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1280/2550] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:19.120712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c0a-e5d3-4843-842d-add5d52d16e0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:19.121682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c0a-e5d3-4843-842d-add5d52d16e0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:19.121682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c0a-e5d3-4843-842d-add5d52d16e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:19.176775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-99c30c0a-e5d3-4843-842d-add5d52d16e0 None None] MFA Rules not processed for user `cd30eb171a8040b4be937d568f0a4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:19.178772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-99c30c0a-e5d3-4843-842d-add5d52d16e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:19.306993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1271/2551] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:19.318497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:19.319989 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:19.425031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265210f0-446e-4906-9dd6-606d7f396430 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:19.429814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:19.430725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:19.431288 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:19.431699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:19.457503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:19.459115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:19.475509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1281/2552] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:21.020394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:21.021000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:21.021000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:21.041554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] MFA Rules not processed for user `cd30eb171a8040b4be937d568f0a4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:21.042932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:21.051351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1272/2553] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:22.243804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:22.245323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:22.261217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6dc43-9c35-4f44-98c6-a76e62e9c9bd service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:22.264878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:22.265574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:22.265852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:22.266127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:22.268482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:22.282057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:22.283666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:22.295312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1282/2554] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:22:26.332496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:26.334053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:26.429657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0691a-c0c7-4688-92ec-53f1a4957647 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:26.436904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:26.437396 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:26.437880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:26.438082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:26.438366 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:26.438499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:26.440453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:26.452311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da000-e8ad-4601-b44d-49438003126e service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:26.454240 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:26.456422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:26.460009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:26.460926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:26.461205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:26.461580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] PATH_INFO: `/v3/users/f705392900ef4f158b3bbc6f910fd67b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:26.463710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f705392900ef4f158b3bbc6f910fd67b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:26.465433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:26.470729 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1273/2555] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:26.526554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f705392900ef4f158b3bbc6f910fd67b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.534194 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f705392900ef4f158b3bbc6f910fd67b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.534890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f705392900ef4f158b3bbc6f910fd67b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.546925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f705392900ef4f158b3bbc6f910fd67b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.555155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-9fd77975-a708-4df1-aec6-4d18ac75c797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f705392900ef4f158b3bbc6f910fd67b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.562021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1283/2556] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:22:26 2022] DELETE /identity/v3/users/f705392900ef4f158b3bbc6f910fd67b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:22:26.769867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:26.787950 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23ea5-4cd3-4be4-883c-eddfb8f3a259 service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:26.857439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:26.875602 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:26.876751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:26.877029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:26.877357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] PATH_INFO: `/v3/projects/c7d349f62c0c4747bd1748ca0a4bde00` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:26.878975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d349f62c0c4747bd1748ca0a4bde00)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:26.881214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:26.946853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d349f62c0c4747bd1748ca0a4bde00', 'request_id': 'req-6e02ddce-1364-4542-8fc1-99175fa03335'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.958202 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d349f62c0c4747bd1748ca0a4bde00', 'request_id': 'req-6e02ddce-1364-4542-8fc1-99175fa03335'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.968517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d349f62c0c4747bd1748ca0a4bde00', 'request_id': 'req-6e02ddce-1364-4542-8fc1-99175fa03335'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.980517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] The token cache is being invalidate because project c7d349f62c0c4747bd1748ca0a4bde00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:22:26.980804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.981467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-6e02ddce-1364-4542-8fc1-99175fa03335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:26.983072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1274/2557] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:22:26 2022] DELETE /identity/v3/projects/c7d349f62c0c4747bd1748ca0a4bde00 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:26.988231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:26.988519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:26.988799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.007622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:27.008853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:27.066292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:27.088383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1284/2558] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:27.096526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.096809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.097082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.121044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:27.122304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:27.132079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1275/2559] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:27.136558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.147003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de541b-b7e9-4f6e-a3b3-7c1bbac37f4b None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.152833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.153506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.153783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.154053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.154790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:22:27.155095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.157420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.175589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1285/2560] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:22:27.180700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.191398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf38d01-3e85-4ee3-b210-56527daa41f9 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.197446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.198138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.198472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.198746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.199509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.201594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.231660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1276/2561] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:22:27 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:22:27.237003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.248082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba531285-f51e-4d93-8a69-25081fe8627c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.320811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.321414 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.321690 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.321961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.322665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.324734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.347586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1286/2562] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:22:27 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:22:27.352955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.370975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42fe6b-c3c8-46c7-9d54-fc46003e88cf admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.376804 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.377617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.378062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.378503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.379422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.381350 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.395719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1277/2563] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:27.400403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.411586 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4dd7-41fe-42d7-9263-494d738ab2c6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.417197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.418447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.418447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.418447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.418859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.420665 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8df7da-2fff-49ff-888d-9f4202c39f91 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.441603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1287/2564] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:27.442996 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.454300 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab378c38-1e70-442f-8f2e-7bf61a6e58b3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.459939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.461343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.461508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.461784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] PATH_INFO: `/v3/projects/5cbfc2205718468babfcb65ff7586480/users/bfc20091ae3348d584d73de90b70a039/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.479069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbfc2205718468babfcb65ff7586480, user_id=bfc20091ae3348d584d73de90b70a039, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.481866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.497676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1278/2565] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:27 2022] PUT /identity/v3/projects/5cbfc2205718468babfcb65ff7586480/users/bfc20091ae3348d584d73de90b70a039/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:22:27.503026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde14fa3-ad46-4e1c-b5de-2300fcc9cec5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.503310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde14fa3-ad46-4e1c-b5de-2300fcc9cec5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.503625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde14fa3-ad46-4e1c-b5de-2300fcc9cec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.525157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-bde14fa3-ad46-4e1c-b5de-2300fcc9cec5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:27.526388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bde14fa3-ad46-4e1c-b5de-2300fcc9cec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:27.599629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1288/2566] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:22:27.610280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:27.612615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:27.705643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba9893-0833-482e-b7f6-07aef23c139d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:27.711092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:27.711880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:27.712225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:27.712809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:27.728264 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:27.730152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:27.742574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1279/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:28.544844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:28.545503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:28.545503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:28.576733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] MFA Rules not processed for user `bfc20091ae3348d584d73de90b70a039`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:28.578065 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:28.624239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:28.655704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1289/2568] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:28.659523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:28.670563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7dfa5-5e02-4a2a-ac0e-49b83f53e9b0 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:28.677052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:28.677742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:28.678045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:28.678345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:28.679042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:28.681156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:28.697194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:28.716294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1280/2569] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:22:28 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:22:28.717507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:28.729673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a939fa8-4285-4b8b-94c5-08b1f781a995 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:28.796058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:28.796736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:28.797090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:28.797372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:28.798174 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:28.800228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:28.824806 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1290/2570] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:22:28 2022] POST /identity/v3/users => generated 309 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:22:28.829585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:28.845531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ea7fd-9a33-4d82-9e7b-2f9ed623ca4e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:28.854042 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:28.855004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:28.855390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:28.855838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:28.856884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:28.858939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:28.871454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1281/2571] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:28.876233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:28.887211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6bd55-7e94-4162-ae7e-3d242566ba34 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:28.892134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:28.893089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:28.893397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:28.893692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] PATH_INFO: `/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a/users/09b995273878444d8f9aee5326597bce/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:28.910483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f217780b8dd4c7f9516fc7513c94b7a, user_id=09b995273878444d8f9aee5326597bce, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:28.913298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:28.928814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1291/2572] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:28 2022] PUT /identity/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a/users/09b995273878444d8f9aee5326597bce/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:28.933669 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:28.945417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c111ca6-3608-49ee-8b9a-6e6be71e6aa7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:29.007814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:29.008659 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:29.009026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:29.009305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:29.010355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:29.013004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89be3d-6b20-498f-b7d2-f241aaa282f0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:29.028672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1282/2573] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:29.032984 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:29.044471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98db9f-1547-470f-b22b-ed3c9a88a7c6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:29.050082 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:29.051023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:29.051303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:29.051626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] PATH_INFO: `/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a/users/09b995273878444d8f9aee5326597bce/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:29.053792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f217780b8dd4c7f9516fc7513c94b7a, user_id=09b995273878444d8f9aee5326597bce, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:29.056639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:29.069768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1292/2574] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:29 2022] PUT /identity/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a/users/09b995273878444d8f9aee5326597bce/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:29.911140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:29.912062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:29.912062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:29.943613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] MFA Rules not processed for user `09b995273878444d8f9aee5326597bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:29.945009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:30.055343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1283/2575] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:30.769962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:30.771769 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:30.869283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabc23-6c20-4384-bb17-3af5b0feeacb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:30.875682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:30.876399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:30.876683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:30.877517 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:30.879420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:30.891805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:30.893442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:30.904842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1293/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:31.431105 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:31.510228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812e5e-e344-43cf-998d-1c634fdbfe13 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:31.516137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:31.517106 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:31.517391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:31.517700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] PATH_INFO: `/v3/users/bfc20091ae3348d584d73de90b70a039` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:31.519063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc20091ae3348d584d73de90b70a039)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:31.521258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:31.527468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:31.591616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc20091ae3348d584d73de90b70a039'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:31.600604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc20091ae3348d584d73de90b70a039'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:31.615442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc20091ae3348d584d73de90b70a039'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:31.616269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc20091ae3348d584d73de90b70a039'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:31.625399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc20091ae3348d584d73de90b70a039'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:31.633189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1284/2577] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:22:31 2022] DELETE /identity/v3/users/bfc20091ae3348d584d73de90b70a039 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:31.911916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:31.923437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56163e7f-ec96-4dd4-a7ad-32106e3873cf service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:31.950247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:31.994719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:31.995862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:31.996307 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:31.996589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] PATH_INFO: `/v3/projects/5cbfc2205718468babfcb65ff7586480` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:31.997975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbfc2205718468babfcb65ff7586480)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:32.000104 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:32.050068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbfc2205718468babfcb65ff7586480', 'request_id': 'req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.057435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbfc2205718468babfcb65ff7586480', 'request_id': 'req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.064170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbfc2205718468babfcb65ff7586480', 'request_id': 'req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.078273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] The token cache is being invalidate because project 5cbfc2205718468babfcb65ff7586480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:22:32.078618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.079252 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.081130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1294/2578] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:22:31 2022] DELETE /identity/v3/projects/5cbfc2205718468babfcb65ff7586480 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:32.085368 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:32.087126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:32.104339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e817d8-89be-4b6d-a839-f211411e9816 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:32.182261 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:32.183960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.184246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.184551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] PATH_INFO: `/v3/users/09b995273878444d8f9aee5326597bce` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.186334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b995273878444d8f9aee5326597bce)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:32.188361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:32.245280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b995273878444d8f9aee5326597bce'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.254122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b995273878444d8f9aee5326597bce'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.269429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b995273878444d8f9aee5326597bce'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.270474 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b995273878444d8f9aee5326597bce'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.278664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1d197871-0997-4a29-8e91-59f61d03f0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b995273878444d8f9aee5326597bce'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.286898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1285/2579] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:22:32 2022] DELETE /identity/v3/users/09b995273878444d8f9aee5326597bce => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:32.495281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:32.496896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:32.512387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0c743-63d8-48ac-a9a4-e0f437bd4ec8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:32.577415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:32.578289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.578574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.578873 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] PATH_INFO: `/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.580179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f217780b8dd4c7f9516fc7513c94b7a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:32.582115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:32.636228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f217780b8dd4c7f9516fc7513c94b7a', 'request_id': 'req-d560970a-7a07-4a34-9108-7aa0bd7011aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.642802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f217780b8dd4c7f9516fc7513c94b7a', 'request_id': 'req-d560970a-7a07-4a34-9108-7aa0bd7011aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.644052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:32.650842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f217780b8dd4c7f9516fc7513c94b7a', 'request_id': 'req-d560970a-7a07-4a34-9108-7aa0bd7011aa'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.665526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] The token cache is being invalidate because project 0f217780b8dd4c7f9516fc7513c94b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:22:32.665811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.666424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d560970a-7a07-4a34-9108-7aa0bd7011aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:22:32.668029 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1295/2580] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:22:32 2022] DELETE /identity/v3/projects/0f217780b8dd4c7f9516fc7513c94b7a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:32.673645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.674158 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.674324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.681445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:32.696286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:32.698047 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:32.765214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1286/2581] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:32.773649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.773926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.774190 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.799425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:32.800706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:32.811072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1296/2582] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:32.815940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:32.826535 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea635c-2b09-4425-afa6-9e39760db4a7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:32.832179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:32.832934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.833258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.833530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.834271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:22:32.834579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:32.836276 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:32.855335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1287/2583] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:22:32.861234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:32.874130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666b1c2-6c31-4929-9617-ffb220e27ae4 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:32.879640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:32.880334 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:32.880618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:32.880896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:32.881601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:32.883574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:32.916601 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1297/2584] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:22:32 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:22:32.921355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:32.932424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a0fe-e3d4-4a3a-9dbf-6b8c24ec46e7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.018732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.020083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.020682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.021282 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.022769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.026294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.051952 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1288/2585] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:22:32 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:22:33.057671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.069402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb443-5e26-4d2a-a8b7-47a5c7f603fa admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.077240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.077963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.078259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.078516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.079299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.081072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.099138 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1298/2586] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:22:33.103211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.115030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcb34c-e4c5-4f30-9b4d-113873d0048a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.123076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.124587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.124587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.124587 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.125190 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.126907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555ee1b-6981-4f09-9229-0514a6e4e2c3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.149693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1289/2587] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:33.156724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.167565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010060bc-734f-4563-a62e-bef2a298d613 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.172789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.173732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.174039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.174860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] PATH_INFO: `/v3/projects/e37eea7f398a41b5a7e08c5bda27c1dd/users/8257aa07418b469683a2f65901469ff9/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.192519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37eea7f398a41b5a7e08c5bda27c1dd, user_id=8257aa07418b469683a2f65901469ff9, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.195247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.210112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1299/2588] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:33 2022] PUT /identity/v3/projects/e37eea7f398a41b5a7e08c5bda27c1dd/users/8257aa07418b469683a2f65901469ff9/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:33.222230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.222520 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.222802 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.250710 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] MFA Rules not processed for user `8257aa07418b469683a2f65901469ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:33.251984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:33.319123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:33.330895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1290/2589] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:33.331962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.402993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bcc5f-1a00-4201-98c2-1b3ddbf852c1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.408868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.409591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.409897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.410133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.410855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.412908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.445200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1300/2590] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:22:33 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:22:33.449923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.462842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3381a9-8258-49da-97d3-eae9efac40fc None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.534567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.535255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.535569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.535840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.536550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.538588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.561900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1291/2591] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:22:33 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:22:33.566032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.577675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebb25b-e447-419c-a978-c988c7e8c882 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.582612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.583309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.583626 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.583900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.584632 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.586257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.597703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1301/2592] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:33.602137 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.614003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842eb85-4264-4126-93cd-1fb79f0ded43 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.619754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.620694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.621004 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.621284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] PATH_INFO: `/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a/users/8c8ea74aab37490290c9ebad5afb49b6/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.641039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c4f31cf7ef4ecc98a6967937acc91a, user_id=8c8ea74aab37490290c9ebad5afb49b6, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.644391 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.661709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1292/2593] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:33 2022] PUT /identity/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a/users/8c8ea74aab37490290c9ebad5afb49b6/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:22:33.667423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.681590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fe3be-d03a-47ee-a720-a7dc418377f6 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.750036 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:33.761521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.762506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.762874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.763763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.765056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.766733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b304b2-f7ca-4978-acf8-170235f3e17d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.778009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1302/2594] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:22:33.782049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.793819 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b67288-b445-48ce-94c1-78e0398909dd admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:33.798827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:33.799895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.800360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.800564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] PATH_INFO: `/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a/users/8c8ea74aab37490290c9ebad5afb49b6/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.802691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c4f31cf7ef4ecc98a6967937acc91a, user_id=8c8ea74aab37490290c9ebad5afb49b6, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:33.805575 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:33.818411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1293/2595] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:22:33 2022] PUT /identity/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a/users/8c8ea74aab37490290c9ebad5afb49b6/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:22:33.829197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:33.829488 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:33.829768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:33.863662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] MFA Rules not processed for user `8c8ea74aab37490290c9ebad5afb49b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:33.864900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:33.944067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1303/2596] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:33.955498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:33.957488 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:34.031826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44a2f-2ce7-4348-9bfa-4f61f8ec4807 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:34.035699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:34.036519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:34.036807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:34.037087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:34.097611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:34.099102 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a9007-edc0-42f7-9ed3-b822acc5c45e service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:34.108260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1294/2597] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:36.464822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:36.466973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:36.543708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260544ed-50fe-474d-a2be-d32f544f761e None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:36.549705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:36.550374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:36.550657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:36.550962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:36.552831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:36.569624 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:36.571342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:36.582938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1304/2598] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:22:39.790749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:39.791670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:39.791670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:39.801011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:39.819289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] MFA Rules not processed for user `cd30eb171a8040b4be937d568f0a4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:22:39.821632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:39.842300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:22:39.843832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:22:39.858254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241339-d5ad-4419-bf72-f401eea14ec1 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:22:39.864725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:22:39.865413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:22:39.865701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:22:39.866412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:22:39.876573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:22:39.878027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:22:39.888877 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1305/2599] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:22:39.906688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1295/2600] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:22:49.884086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=79 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:22:49.897938 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=23 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:02.259914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:02.261572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:02.323050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac9bb-a832-40c6-a1fd-2fb6ec8d4ead service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:02.326834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:02.327514 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:02.327812 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:02.328088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:02.329964 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:02.395031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:02.396431 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee0c7a-4a18-4c5a-a20c-c92ab373060d service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:02.405884 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1306/2601] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:05.435666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:05.526639 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e300-d9d9-4e4f-a2f0-42cf47bb8a60 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:05.531634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:05.532726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:05.532904 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:05.533077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] PATH_INFO: `/v3/users/8257aa07418b469683a2f65901469ff9` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:05.534418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8257aa07418b469683a2f65901469ff9)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:05.536441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:05.622115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8257aa07418b469683a2f65901469ff9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:05.641647 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8257aa07418b469683a2f65901469ff9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:05.660440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8257aa07418b469683a2f65901469ff9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:05.661433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8257aa07418b469683a2f65901469ff9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:05.669939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8257aa07418b469683a2f65901469ff9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:05.676551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1296/2602] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:05 2022] DELETE /identity/v3/users/8257aa07418b469683a2f65901469ff9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:05.681208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:05.681765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:05.681765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:05.701254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:05.702584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:05.775620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1307/2603] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:05.787424 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:05.790469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:05.903983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6204dd-df9a-4b39-9db0-192147fd90b9 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:05.907414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:05.908633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:05.909126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:05.909616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:05.928110 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:05.930327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:05.946173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1297/2604] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:06.346353 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:06.353456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:06.369831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5462be-98aa-49df-bc5f-7bb700eb358c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:06.378876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:06.379825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:06.380262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:06.380549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] PATH_INFO: `/v3/projects/e37eea7f398a41b5a7e08c5bda27c1dd` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:06.381928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37eea7f398a41b5a7e08c5bda27c1dd)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:06.384346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:06.443830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37eea7f398a41b5a7e08c5bda27c1dd', 'request_id': 'req-3eb4ca56-7493-4612-8c79-7531a3e808e4'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.445198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:06.455662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37eea7f398a41b5a7e08c5bda27c1dd', 'request_id': 'req-3eb4ca56-7493-4612-8c79-7531a3e808e4'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.463258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37eea7f398a41b5a7e08c5bda27c1dd', 'request_id': 'req-3eb4ca56-7493-4612-8c79-7531a3e808e4'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.480378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] The token cache is being invalidate because project e37eea7f398a41b5a7e08c5bda27c1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:06.480833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.481770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.484291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1308/2605] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:06 2022] DELETE /identity/v3/projects/e37eea7f398a41b5a7e08c5bda27c1dd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:06.489318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:06.500724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4b043-062b-4c39-8067-31da6fad2aad service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:06.532410 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:06.604614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:06.605663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:06.606111 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:06.606528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] PATH_INFO: `/v3/users/8c8ea74aab37490290c9ebad5afb49b6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:06.608702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8ea74aab37490290c9ebad5afb49b6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:06.611414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:06.696458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8ea74aab37490290c9ebad5afb49b6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.707641 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8ea74aab37490290c9ebad5afb49b6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.724233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8ea74aab37490290c9ebad5afb49b6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.725153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8ea74aab37490290c9ebad5afb49b6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.734673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-191021bc-07f2-4349-96eb-73d4d842a3f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8ea74aab37490290c9ebad5afb49b6'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:06.743194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1298/2606] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:06 2022] DELETE /identity/v3/users/8c8ea74aab37490290c9ebad5afb49b6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:07.224963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.226590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:07.240783 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4ca56-7493-4612-8c79-7531a3e808e4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.311891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.312763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.313037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.313309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] PATH_INFO: `/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.314513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c4f31cf7ef4ecc98a6967937acc91a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.316704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.377614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c4f31cf7ef4ecc98a6967937acc91a', 'request_id': 'req-901e0333-1aca-4453-9990-dbe1090d83a5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:07.384335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c4f31cf7ef4ecc98a6967937acc91a', 'request_id': 'req-901e0333-1aca-4453-9990-dbe1090d83a5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:07.390482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c4f31cf7ef4ecc98a6967937acc91a', 'request_id': 'req-901e0333-1aca-4453-9990-dbe1090d83a5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:07.404973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] The token cache is being invalidate because project 44c4f31cf7ef4ecc98a6967937acc91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:07.405278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:07.405967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-901e0333-1aca-4453-9990-dbe1090d83a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:07.407549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1309/2607] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:07 2022] DELETE /identity/v3/projects/44c4f31cf7ef4ecc98a6967937acc91a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:07.416651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.417144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.417485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.439633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:07.440910 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:07.506059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1299/2608] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:07.515052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.515335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.515650 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.535553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:07.536689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:07.544864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1310/2609] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:07.549534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.561733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7837b3-90f4-496f-93a8-3132c56c7553 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.567959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.568646 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.568927 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.569199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.569942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:07.570247 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.572026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.585582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1300/2610] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:23:07.592037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.606874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5a1a5-229d-40ac-8281-09fbeb3356e0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.614419 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.615131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.615400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.615731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.616462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.618392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.647239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1311/2611] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:07 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:07.653079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.665312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc727b4-807e-4966-ad76-a2ad447d72b5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.701340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:07.745268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.745986 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.746271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.746550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.747259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.749257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.775213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1301/2612] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:07 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:07.779904 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.791760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfe6e8-4ef4-4e69-8b50-39984ce9db35 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.798483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.799145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.799430 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.799739 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.800605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.802217 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.813716 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1312/2613] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:07.818743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.830059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d0bab-62e0-4392-98b0-2e6b03b65b93 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.836059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.836744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.837030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.837307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.838048 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.839777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9feb65-f2f2-43fa-ab40-724f6feacb49 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.851642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1302/2614] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:07.857257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:07.871380 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e21f9-2f9f-4321-b366-2272150c7ea7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:07.877317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:07.878669 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.879057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.879558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] PATH_INFO: `/v3/projects/0efb6b864a0f445b9ac3b65ce116d89a/users/6ca208c97a4246909012418ec3131bc8/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.899705 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efb6b864a0f445b9ac3b65ce116d89a, user_id=6ca208c97a4246909012418ec3131bc8, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:07.903381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:07.923318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1313/2615] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:07 2022] PUT /identity/v3/projects/0efb6b864a0f445b9ac3b65ce116d89a/users/6ca208c97a4246909012418ec3131bc8/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:07.942128 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:07.942604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:07.943077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:07.975363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] MFA Rules not processed for user `6ca208c97a4246909012418ec3131bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:07.976699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:08.054908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1303/2616] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:08.064760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.075858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:08.150967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663c5cb-62cb-45a8-9156-d2e75ac8fb88 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.156149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.157158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.157559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.157910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.158923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.161270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.189706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1314/2617] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:08 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:08.194118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.205800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce1d64-d604-4da0-911d-6d281329ac8d None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.267015 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.267757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.268043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.268316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.269238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.271246 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.294428 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1304/2618] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:08 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:08.298878 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.311230 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d64b-607d-483d-921e-bb246b48c038 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.316394 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.317090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.317374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.317638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.318383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.320073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.332285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1315/2619] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:08.336827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.348174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8a343-b1a6-4902-af9c-18907e95dc70 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.354036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.354929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.355222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.355521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] PATH_INFO: `/v3/projects/ab139e7431a844e6901d4810c520cffc/users/0d3b8efb0c0542eba982c6abb7235dd8/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.375020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab139e7431a844e6901d4810c520cffc, user_id=0d3b8efb0c0542eba982c6abb7235dd8, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.377751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.399251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1305/2620] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:08 2022] PUT /identity/v3/projects/ab139e7431a844e6901d4810c520cffc/users/0d3b8efb0c0542eba982c6abb7235dd8/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:08.404610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.416192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde17bb-b49d-4ce7-908f-3de40190bdf0 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.515326 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.516078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.516572 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.517637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.518832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.520674 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853fa8-0d65-4239-be7f-fcf93e31ab7f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.535524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1316/2621] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:08.537650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.551550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:08.557299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6250d6b-fb9d-4697-bc8d-62781b48018e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.563837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.564860 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.565144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.565419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] PATH_INFO: `/v3/projects/ab139e7431a844e6901d4810c520cffc/users/0d3b8efb0c0542eba982c6abb7235dd8/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.567586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab139e7431a844e6901d4810c520cffc, user_id=0d3b8efb0c0542eba982c6abb7235dd8, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.570361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.582188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1306/2622] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:08 2022] PUT /identity/v3/projects/ab139e7431a844e6901d4810c520cffc/users/0d3b8efb0c0542eba982c6abb7235dd8/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:08.592819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.593134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.593417 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.623146 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] MFA Rules not processed for user `0d3b8efb0c0542eba982c6abb7235dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:08.624727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:08.699019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:08.703704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1317/2623] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:08.717584 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:08.719414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:08.782468 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672643a3-be68-4158-b833-5f62fc34a09e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:08.786704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:08.787380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:08.787694 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:08.787970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:08.798220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:08.799736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:08.811021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1307/2624] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:09.196615 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:09.206900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3789d5-e17d-4c8e-af2e-a7fa28ca2125 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:09.212853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3b8efb0c0542eba982c6abb7235dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab139e7431a844e6901d4810c520cffc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:09.213781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:09.214108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:09.214456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:09.215296 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:09.215792 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:09.217293 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:09.230139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1318/2625] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:23:09.234597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:09.245255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d089e1-206f-4eed-bb1e-c13267654015 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:09.250127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3b8efb0c0542eba982c6abb7235dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab139e7431a844e6901d4810c520cffc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:09.250815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:09.251108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:09.251374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:09.252235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:09.254223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:09.285087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1308/2626] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:09 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:09.289410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:09.308102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33930d-7d84-4504-815b-cebf1e35e831 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:09.383610 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3b8efb0c0542eba982c6abb7235dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab139e7431a844e6901d4810c520cffc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:09.384526 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:09.385711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:09.386420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:09.387660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:09.388083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:09.390033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:09.408324 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1319/2627] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:23:09.443168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:09.454542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f3f6c-f2e3-4940-baaa-c18d62e2e798 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:09.459718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d3b8efb0c0542eba982c6abb7235dd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab139e7431a844e6901d4810c520cffc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:09.460629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:09.460908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:09.461185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] PATH_INFO: `/v3/projects/fbb74336e4014ef7b6f5a234a5191225` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:09.463319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fbb74336e4014ef7b6f5a234a5191225)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:09.464508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:09.544534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb74336e4014ef7b6f5a234a5191225', 'request_id': 'req-2d93ff08-6f91-4568-a12a-4b62758240e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:09.558418 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb74336e4014ef7b6f5a234a5191225', 'request_id': 'req-2d93ff08-6f91-4568-a12a-4b62758240e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:09.570514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb74336e4014ef7b6f5a234a5191225', 'request_id': 'req-2d93ff08-6f91-4568-a12a-4b62758240e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:09.590249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] The token cache is being invalidate because project fbb74336e4014ef7b6f5a234a5191225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:09.590548 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:09.591926 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2d93ff08-6f91-4568-a12a-4b62758240e0 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:09.593812 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1309/2628] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:09 2022] DELETE /identity/v3/projects/fbb74336e4014ef7b6f5a234a5191225 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:09.846646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:09.848584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:09.967631 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a0add-e56c-41ed-b5a4-5a8480c627b9 tempest-VolumeQuotaClassesTest-1756288601 tempest-VolumeQuotaClassesTest-1756288601-project-admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:10.085853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:10.086666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:10.086954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:10.087222 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] PATH_INFO: `/v3/users/6ca208c97a4246909012418ec3131bc8` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:10.088645 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca208c97a4246909012418ec3131bc8)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:10.090653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:10.138383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:10.170608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca208c97a4246909012418ec3131bc8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:10.180795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca208c97a4246909012418ec3131bc8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:10.181560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca208c97a4246909012418ec3131bc8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:10.206070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca208c97a4246909012418ec3131bc8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:10.216047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca208c97a4246909012418ec3131bc8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:10.224012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1320/2629] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:09 2022] DELETE /identity/v3/users/6ca208c97a4246909012418ec3131bc8 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:10.229007 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:10.229967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:10.230332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:10.248272 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:10.249436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:10.310556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:10.331387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1310/2630] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:10.344757 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:10.346363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:10.473291 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc9610-96f0-4d1c-8a22-646fdf9ef2d9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:10.484005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:10.484694 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:10.484971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:10.485243 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:10.496390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:10.498407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:10.513942 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1321/2631] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:10.965592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:10.967746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:10.993702 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802d5ae-a837-4a1d-b1ae-df3b07394ad5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:11.002461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:11.003999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:11.004653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:11.005113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] PATH_INFO: `/v3/projects/0efb6b864a0f445b9ac3b65ce116d89a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:11.006979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efb6b864a0f445b9ac3b65ce116d89a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:11.009852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:11.085940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efb6b864a0f445b9ac3b65ce116d89a', 'request_id': 'req-591cb5c0-661e-4099-9b22-01315b974db1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.098506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efb6b864a0f445b9ac3b65ce116d89a', 'request_id': 'req-591cb5c0-661e-4099-9b22-01315b974db1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.106632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efb6b864a0f445b9ac3b65ce116d89a', 'request_id': 'req-591cb5c0-661e-4099-9b22-01315b974db1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.121768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] The token cache is being invalidate because project 0efb6b864a0f445b9ac3b65ce116d89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:11.122328 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.123142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.125183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1311/2632] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:10 2022] DELETE /identity/v3/projects/0efb6b864a0f445b9ac3b65ce116d89a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:11.129315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:11.140400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1b5c-4dd4-4dc1-9eb0-06212a6b06ca service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:11.242066 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:11.244001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:11.244487 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:11.245001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] PATH_INFO: `/v3/users/0d3b8efb0c0542eba982c6abb7235dd8` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:11.247357 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3b8efb0c0542eba982c6abb7235dd8)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:11.250019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:11.364306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3b8efb0c0542eba982c6abb7235dd8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.388692 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:11.395156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3b8efb0c0542eba982c6abb7235dd8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.397465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3b8efb0c0542eba982c6abb7235dd8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.462713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3b8efb0c0542eba982c6abb7235dd8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.479725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3b8efb0c0542eba982c6abb7235dd8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:11.488263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1322/2633] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:11 2022] DELETE /identity/v3/users/0d3b8efb0c0542eba982c6abb7235dd8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:11.885084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:11.888563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:11.907889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cb5c0-661e-4099-9b22-01315b974db1 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:11.993931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:11.994859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:11.995104 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:11.995388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] PATH_INFO: `/v3/projects/ab139e7431a844e6901d4810c520cffc` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:11.996745 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab139e7431a844e6901d4810c520cffc)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:11.998763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.059682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab139e7431a844e6901d4810c520cffc', 'request_id': 'req-3272c93d-896b-456b-97bb-336e732bd66c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:12.070377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab139e7431a844e6901d4810c520cffc', 'request_id': 'req-3272c93d-896b-456b-97bb-336e732bd66c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:12.086707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab139e7431a844e6901d4810c520cffc', 'request_id': 'req-3272c93d-896b-456b-97bb-336e732bd66c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:12.117320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.120409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:12.128652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] The token cache is being invalidate because project ab139e7431a844e6901d4810c520cffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:12.130842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:12.133503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-3272c93d-896b-456b-97bb-336e732bd66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:12.137984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1312/2634] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:11 2022] DELETE /identity/v3/projects/ab139e7431a844e6901d4810c520cffc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:12.153571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11db3-8ed3-488f-92d7-8b8341bc6ddf None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.154285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11db3-8ed3-488f-92d7-8b8341bc6ddf None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.154671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11db3-8ed3-488f-92d7-8b8341bc6ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.193859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:12.207621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-adb11db3-8ed3-488f-92d7-8b8341bc6ddf None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:12.209780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-adb11db3-8ed3-488f-92d7-8b8341bc6ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:12.276758 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540aec0-b303-41a1-b3b7-0d1293887fef admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.311935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1313/2635] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:12.314875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.314875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.314875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.333383 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:12.334191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:12.334623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.335393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.335804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.336162 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.338163 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:12.343619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1314/2636] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:12.347862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.351917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.353621 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.357684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d4e39-ef38-44c3-9570-ee64600e1e00 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.362968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.363968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.364360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.364636 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.365630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:12.365907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1323/2637] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:12.366506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.367824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.399496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1315/2638] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:23:12.400711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.414087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a60ef-c12c-46a8-a391-b456dada52b1 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.425428 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.425821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.426105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.426381 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.427097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.429118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.477329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.483744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1324/2639] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:12 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:12.498054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ca4ae-a178-4730-b5f6-ed5ed7415da6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.599656 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.600865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.601364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.601866 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.603242 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.607209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.640148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1316/2640] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:12 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:12.645195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.667089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56c339-b50e-4d93-b986-baca5a3a2707 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.679420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.681850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.682744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.683756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.685799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.689421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.718703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1325/2641] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:12.720603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.740204 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855ce5-3503-4b3c-873c-cb175249b41a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.749521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.751068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.751580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.752046 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.753396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.756187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c077-f308-4203-9986-620aa4372f0e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.775061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1317/2642] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:12.776647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:12.796611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c6b9f-736d-4302-87ba-e44b38857804 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:12.803533 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:12.804564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.804939 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.805383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] PATH_INFO: `/v3/projects/990185f8f2eb4deeb6d2427fd9f0b591/users/ce103ed785d14f3bb144c3bcc4a65492/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.844147 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990185f8f2eb4deeb6d2427fd9f0b591, user_id=ce103ed785d14f3bb144c3bcc4a65492, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:12.849261 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:12.875521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1326/2643] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:12 2022] PUT /identity/v3/projects/990185f8f2eb4deeb6d2427fd9f0b591/users/ce103ed785d14f3bb144c3bcc4a65492/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:12.894855 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:12.895159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:12.895534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:12.934579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] MFA Rules not processed for user `ce103ed785d14f3bb144c3bcc4a65492`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:12.936148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:13.014482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:13.029002 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1318/2644] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:13.049646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:13.052585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:13.107524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:13.156538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0b34e-cf9d-409d-990d-2b8d1e9086c2 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:13.162073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:13.162801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:13.163680 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:13.163970 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:13.184983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:13.186554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e02a86-3d73-4f9f-8c00-43aee0876b2d service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:13.217157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1327/2645] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:13.234596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:13.373765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7b509-4e0a-49d1-8a64-8748154215a6 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:13.381441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:13.382511 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:13.382889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:13.383262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] PATH_INFO: `/v3/users/ce103ed785d14f3bb144c3bcc4a65492` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:13.384967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce103ed785d14f3bb144c3bcc4a65492)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:13.387315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:13.478307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce103ed785d14f3bb144c3bcc4a65492'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:13.489572 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce103ed785d14f3bb144c3bcc4a65492'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:13.508249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce103ed785d14f3bb144c3bcc4a65492'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:13.510188 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce103ed785d14f3bb144c3bcc4a65492'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:13.533177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce103ed785d14f3bb144c3bcc4a65492'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:13.544205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1319/2646] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:13 2022] DELETE /identity/v3/users/ce103ed785d14f3bb144c3bcc4a65492 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:13.550032 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:13.550706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:13.550706 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:13.585309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:13.587699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:13.678432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1328/2647] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:13.696660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:13.697703 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:13.798676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b0b7-d38e-4860-94b0-7381ecfb99ca admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:13.803380 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:13.804209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:13.804796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:13.805319 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:13.818288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:13.820195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf34f1c-3328-475a-aee8-27e4315cced7 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:13.833252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1320/2648] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:14.476646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:14.478642 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:14.504474 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b5c6f-07f2-487c-b37a-134a9929acc8 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:14.516895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:14.517821 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:14.518115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:14.518391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] PATH_INFO: `/v3/projects/990185f8f2eb4deeb6d2427fd9f0b591` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:14.519765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990185f8f2eb4deeb6d2427fd9f0b591)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:14.522165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:14.581830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990185f8f2eb4deeb6d2427fd9f0b591', 'request_id': 'req-e9e203d5-83be-48a6-971a-b39a9823e2b7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:14.590759 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990185f8f2eb4deeb6d2427fd9f0b591', 'request_id': 'req-e9e203d5-83be-48a6-971a-b39a9823e2b7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:14.598729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990185f8f2eb4deeb6d2427fd9f0b591', 'request_id': 'req-e9e203d5-83be-48a6-971a-b39a9823e2b7'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:14.618704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] The token cache is being invalidate because project 990185f8f2eb4deeb6d2427fd9f0b591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:14.619111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:14.620073 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e9e203d5-83be-48a6-971a-b39a9823e2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:14.622620 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1329/2649] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:14 2022] DELETE /identity/v3/projects/990185f8f2eb4deeb6d2427fd9f0b591 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:14.630651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:14.632466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:14.633255 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:14.662378 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:14.663946 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:14.678990 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:14.755840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1321/2650] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:14.764297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:14.764606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:14.764847 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:14.788043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:14.789318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:14.798365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1330/2651] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:14.802888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:14.815558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fa561-b91e-459a-b405-45b25959864e None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:14.820907 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:14.821756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:14.822141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:14.822517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:14.823429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:14.823881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:14.825649 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:14.849011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1322/2652] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:23:14.853269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:14.864103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487136d-e106-41a9-8143-f4e425b2a51f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:14.870464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:14.871294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:14.871693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:14.871977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:14.872709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:14.875017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:14.917880 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1331/2653] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:14 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:14.924598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:14.938626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed6cf7-7c16-4a2f-bd2b-65faf22c20d4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:15.026973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:15.028460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.028460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.028460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.029043 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:15.031297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:15.063854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1323/2654] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:14 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:15.065894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:15.078591 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b14849-28f7-4d94-b835-47dd7999bdf7 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:15.084648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:15.086053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.087227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.087227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.087920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:15.090377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:15.108619 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1332/2655] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:15.110486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:15.122617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881bdc26-b829-4ac1-acaa-3547851018e4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:15.129336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:15.130356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.131357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.131743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.132643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:15.134533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289800-29b3-4548-bfd9-5005df3dd2ea admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:15.175021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1324/2656] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:15.178232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:15.192974 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c578612-70b2-48ad-92e0-aff8904375b9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:15.199921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:15.201131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.201613 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.201969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] PATH_INFO: `/v3/projects/6f6de2c24f1d409889d26b51329ba089/users/1c875d37ca464377bb72a14959d141de/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.219543 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6de2c24f1d409889d26b51329ba089, user_id=1c875d37ca464377bb72a14959d141de, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:15.222442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:15.225737 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:15.244228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1333/2657] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:15 2022] PUT /identity/v3/projects/6f6de2c24f1d409889d26b51329ba089/users/1c875d37ca464377bb72a14959d141de/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:15.259436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34102-6fbc-4e81-a067-d58bea07c8f1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.260556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34102-6fbc-4e81-a067-d58bea07c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.260970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34102-6fbc-4e81-a067-d58bea07c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.295911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d8d34102-6fbc-4e81-a067-d58bea07c8f1 None None] MFA Rules not processed for user `1c875d37ca464377bb72a14959d141de`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:15.296936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d8d34102-6fbc-4e81-a067-d58bea07c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:15.418810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1325/2658] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:15.429106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:15.430899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:15.514577 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b252-3e76-4b16-8b96-c57b41f46ac4 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:15.518634 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:15.519442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:15.519770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:15.520052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:15.529793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:15.531306 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791a40a-c48f-40e8-80c3-7c8cf4a75bcf service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:15.542445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1334/2659] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:17.058195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:17.059618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:17.059618 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:17.080226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] MFA Rules not processed for user `1c875d37ca464377bb72a14959d141de`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:17.081644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:17.105205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1326/2660] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:17.710722 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:17.711228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:17.711590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:17.760360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:17.762001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:17.916273 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1335/2661] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:17.989538 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:17.993768 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:18.011945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:18.136983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc915cba-0d39-4b74-a29c-2d3f458bb8e0 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:18.150994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:18.152140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:18.152612 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:18.154314 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:18.170088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:18.172563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:18.186441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1327/2662] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:19.138683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:19.140553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:19.237052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d21bfe-384b-4c26-8480-3321aa684223 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:19.245644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:19.246892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:19.248064 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:19.248528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:19.253966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:19.363192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:19.399195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:19.400919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:19.449420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1336/2663] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:23:27.028847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:27.031655 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:27.203324 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc11a-ebc1-4c27-910b-c9a8456012a0 service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:27.211323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:27.212607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:27.214545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:27.215084 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] PATH_INFO: `/v3/users/cf7a76ad651d4f7996116c942103a496` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:27.217735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf7a76ad651d4f7996116c942103a496)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:27.221852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:27.343141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf7a76ad651d4f7996116c942103a496'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.355624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf7a76ad651d4f7996116c942103a496'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.369699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:27.371322 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:27.375217 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf7a76ad651d4f7996116c942103a496'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.376021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf7a76ad651d4f7996116c942103a496'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.386191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf7a76ad651d4f7996116c942103a496'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.394883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1328/2664] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:27 2022] DELETE /identity/v3/users/cf7a76ad651d4f7996116c942103a496 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:27.593852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c6adf-4fe3-4394-a616-5b8bc168440b service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:27.736153 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:27.738412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:27.738412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:27.738412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] PATH_INFO: `/v3/users/b3fe0e12c72c48fcb9ce277ebaa910f3` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:27.741094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fe0e12c72c48fcb9ce277ebaa910f3)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:27.745327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:27.840032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:27.841580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:27.870439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e3e0-7452-4771-afa1-e1b10244f621 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:27.875654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:27.876554 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:27.876840 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:27.877113 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] PATH_INFO: `/v3/projects/994738157bca4d9d887f316699bd9387` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:27.877425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fe0e12c72c48fcb9ce277ebaa910f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.878470 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=994738157bca4d9d887f316699bd9387)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:27.880687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:27.882484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:27.892086 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fe0e12c72c48fcb9ce277ebaa910f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.893335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fe0e12c72c48fcb9ce277ebaa910f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.927165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fe0e12c72c48fcb9ce277ebaa910f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.944921 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fe0e12c72c48fcb9ce277ebaa910f3'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:27.960241 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1337/2665] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:27 2022] DELETE /identity/v3/users/b3fe0e12c72c48fcb9ce277ebaa910f3 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:27.985558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '994738157bca4d9d887f316699bd9387', 'request_id': 'req-78eb010b-b0e7-452a-869e-3af3b871ffb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.003186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '994738157bca4d9d887f316699bd9387', 'request_id': 'req-78eb010b-b0e7-452a-869e-3af3b871ffb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.011973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '994738157bca4d9d887f316699bd9387', 'request_id': 'req-78eb010b-b0e7-452a-869e-3af3b871ffb4'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.031193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] The token cache is being invalidate because project 994738157bca4d9d887f316699bd9387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:28.032152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.033442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.036426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1329/2666] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:27 2022] DELETE /identity/v3/projects/994738157bca4d9d887f316699bd9387 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:28.040501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:28.042220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:28.059423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:28.233223 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59dd6c-6fb1-4abf-829e-ed6e0088e62e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:28.350537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:28.352036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:28.374762 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:28.378475 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:28.379930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:28.380449 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] PATH_INFO: `/v3/users/e8e3067ffbd74e1ea54a2be9fa570dc5` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:28.383094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e3067ffbd74e1ea54a2be9fa570dc5)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:28.385789 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:28.480031 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb010b-b0e7-452a-869e-3af3b871ffb4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:28.487099 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:28.489394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:28.489918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:28.490288 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] PATH_INFO: `/v3/projects/2409e2644eb14d6b9545088c08d22273` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:28.492737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2409e2644eb14d6b9545088c08d22273)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:28.495220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e3067ffbd74e1ea54a2be9fa570dc5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.495377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:28.513640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e3067ffbd74e1ea54a2be9fa570dc5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.514344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e3067ffbd74e1ea54a2be9fa570dc5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.538826 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e3067ffbd74e1ea54a2be9fa570dc5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.551843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8a9874a1-01d6-4566-b464-697c17e3ca00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e3067ffbd74e1ea54a2be9fa570dc5'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.563947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1338/2667] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:28 2022] DELETE /identity/v3/users/e8e3067ffbd74e1ea54a2be9fa570dc5 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:28.619045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2409e2644eb14d6b9545088c08d22273', 'request_id': 'req-dfef3bd0-e905-42e2-9508-56979b4c1cc1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.640297 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2409e2644eb14d6b9545088c08d22273', 'request_id': 'req-dfef3bd0-e905-42e2-9508-56979b4c1cc1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.649181 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2409e2644eb14d6b9545088c08d22273', 'request_id': 'req-dfef3bd0-e905-42e2-9508-56979b4c1cc1'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.664364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] The token cache is being invalidate because project 2409e2644eb14d6b9545088c08d22273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:28.664663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.665475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-dfef3bd0-e905-42e2-9508-56979b4c1cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:28.667533 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1330/2668] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:28 2022] DELETE /identity/v3/projects/2409e2644eb14d6b9545088c08d22273 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:28.677389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:28.677389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:28.677389 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:28.803950 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:28.823805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:28.826557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:28.958810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1339/2669] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:28.968337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:28.969815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:28.970683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.000706 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:29.002008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:29.017701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1331/2670] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:29.019235 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.047756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6571c2-c2e8-4e4e-88c5-63dd446be4b1 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.062599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.063309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.063635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.063962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.064951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:29.065267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.067045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.091871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1340/2671] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:23:29.093434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.131691 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f581d-324b-42a4-8fac-afa972c2c88b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.154815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.159132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.159585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.163615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.164517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.170532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.190092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:29.226373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.239988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1332/2672] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:29 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:23:29.244515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b400-0e67-4c6c-9142-60235de31c2c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.382152 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.384803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.385603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.386282 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.387879 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.392040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.491655 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1341/2673] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:29 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:29.497355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.520226 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba4aa6-9350-4fa6-a352-e78417b3375e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.548876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.553299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:29.562236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.568574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.569213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.576451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.577826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.587199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.591507 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828717ec-8c23-4687-8769-40ee3f38c43e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.600515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.602853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.603745 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.604552 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] PATH_INFO: `/v3/projects/2ef1937b964b497ab2734d543544d5ad` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.607607 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef1937b964b497ab2734d543544d5ad)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.612279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.634709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1333/2674] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:29.637114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.664054 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c08883-452c-4eb3-b170-a8ea2438ec54 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.674605 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.676603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.677299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.678029 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.679411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.685286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.715611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1334/2675] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:23:29.717283 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:29.738714 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4069f-0979-4846-bd40-cf8d9dc8ffe3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:29.745348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef1937b964b497ab2734d543544d5ad', 'request_id': 'req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:29.760178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef1937b964b497ab2734d543544d5ad', 'request_id': 'req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:29.770009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef1937b964b497ab2734d543544d5ad', 'request_id': 'req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:29.793699 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] The token cache is being invalidate because project 2ef1937b964b497ab2734d543544d5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:29.794013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:29.795131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-3ecc870c-42ee-4639-8ba0-6ea4bf021ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:29.797858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1342/2676] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:29 2022] DELETE /identity/v3/projects/2ef1937b964b497ab2734d543544d5ad => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:29.887580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:29.893498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.896545 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.897454 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] PATH_INFO: `/v3/projects/896cc13a4d744c06b80fc97aa4bd1dcf/users/44b32f2ca6d74412b8aaf6bdc15b252a/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.939079 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896cc13a4d744c06b80fc97aa4bd1dcf, user_id=44b32f2ca6d74412b8aaf6bdc15b252a, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:29.942403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:29.963924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1335/2677] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:29 2022] PUT /identity/v3/projects/896cc13a4d744c06b80fc97aa4bd1dcf/users/44b32f2ca6d74412b8aaf6bdc15b252a/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:29.971726 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1177a-eca8-4c87-834b-752706c758cb None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:29.972238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1177a-eca8-4c87-834b-752706c758cb None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:29.972733 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1177a-eca8-4c87-834b-752706c758cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:29.982452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:30.002856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-49a1177a-eca8-4c87-834b-752706c758cb None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:30.004506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-49a1177a-eca8-4c87-834b-752706c758cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:30.122263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1343/2678] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:30.133874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:30.137254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:30.317591 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431e22e-26d2-429b-bc9b-79a6ea5f5ae7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:30.341857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:30.342887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:30.343388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:30.343906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:30.375878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:30.379055 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:30.413168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1336/2679] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:23:37.794276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:37.794887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:37.795220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:37.933200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] MFA Rules not processed for user `44b32f2ca6d74412b8aaf6bdc15b252a`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:37.944249 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:38.104049 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1344/2680] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:38.135517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:38.137149 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:38.299427 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:38.422196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb3c1d-16a9-4b74-ae2d-acdc5665bd9d service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:38.454781 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:38.455705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:38.456220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:38.456567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:38.470131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:38.471943 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083fe2c-fbb9-4aa2-bf80-8c06de576b11 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:38.500547 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1337/2681] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:39.337555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:39.339808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:39.704465 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54084bf-3fb1-413e-8a71-46e5f1e59f25 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:39.710482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:39.717605 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:39.718968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:39.719382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:39.723661 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:39.762960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:39.764943 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e1a90-e414-4bfd-bda2-8def7f6f0071 service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:39.793823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1345/2682] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:41.450223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:41.450719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:41.450824 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:41.478709 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] MFA Rules not processed for user `44b32f2ca6d74412b8aaf6bdc15b252a`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:41.483251 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:41.499617 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1338/2683] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:42.176671 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b54e7-de73-44c7-a7cb-cae4e9bed40f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:42.177425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b54e7-de73-44c7-a7cb-cae4e9bed40f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:42.177425 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b54e7-de73-44c7-a7cb-cae4e9bed40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:42.185113 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:42.230025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-491b54e7-de73-44c7-a7cb-cae4e9bed40f None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:42.231303 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-491b54e7-de73-44c7-a7cb-cae4e9bed40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:42.323132 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1346/2684] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:23:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:42.368648 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:42.370928 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:42.533077 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6add0b8-cefb-4c4e-b0f8-bbfaf781e40f None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:42.547697 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:42.548390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:42.548673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:42.548951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:42.565284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:42.568500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:42.635590 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1339/2685] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:44.180883 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2838-d094-424b-a6c1-1d5d3f97a2a7 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:44.181440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2838-d094-424b-a6c1-1d5d3f97a2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:44.181440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2838-d094-424b-a6c1-1d5d3f97a2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:44.206037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-104c2838-d094-424b-a6c1-1d5d3f97a2a7 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:44.207606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-104c2838-d094-424b-a6c1-1d5d3f97a2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:44.216957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1347/2686] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:44.233913 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:44.247343 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d9418-7a47-4895-9487-5442cd9b3368 service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:44.257623 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:44.258434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:44.258820 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:44.259160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:44.272918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:44.274874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:44.287588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1340/2687] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:44.673268 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a96977-65b3-4253-ae29-a46873febeb2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:44.673558 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a96977-65b3-4253-ae29-a46873febeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:44.673813 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a96977-65b3-4253-ae29-a46873febeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:44.702495 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-54a96977-65b3-4253-ae29-a46873febeb2 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:44.704618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-54a96977-65b3-4253-ae29-a46873febeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:44.716464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1348/2688] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:44.734340 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:44.744857 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e09c38-ba40-4486-9109-36cc69e60623 service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:44.751307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:44.752353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:44.752901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:44.753356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:44.769034 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:44.771032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:44.786599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1341/2689] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:45.265742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a47a-244e-4fa2-9c5a-4a25139b269c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:45.266279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a47a-244e-4fa2-9c5a-4a25139b269c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:45.267178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a47a-244e-4fa2-9c5a-4a25139b269c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:45.294983 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e028a47a-244e-4fa2-9c5a-4a25139b269c None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:45.296427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e028a47a-244e-4fa2-9c5a-4a25139b269c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:45.305781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1349/2690] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:45.319568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:45.332786 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab800-efd4-44a9-80e4-ec3121229eff service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:45.340451 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:45.340744 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:45.340931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:45.341222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:45.353409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:45.355241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:45.370374 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1342/2691] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:45.756185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:45.756505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:45.756750 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:45.781290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:45.782996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:45.810333 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1350/2692] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:45.813233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:45.845630 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab920c-fd7d-4d61-884d-55af43f3e705 service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:45.860115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:45.861253 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:45.861879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:45.862396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:45.879593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:45.881338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae5bb2-6d2b-4a0d-b0df-47f8fe5411ab service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:45.892259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1343/2693] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:23:48.048623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:48.050272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:48.066590 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b872908-0df1-443b-a28c-6aa37388a364 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:48.071676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:48.072576 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:48.072862 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:48.073180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] PATH_INFO: `/v3/users/1c875d37ca464377bb72a14959d141de` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:48.074441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c875d37ca464377bb72a14959d141de)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:48.076442 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:48.135276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c875d37ca464377bb72a14959d141de'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.144618 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c875d37ca464377bb72a14959d141de'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.145545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c875d37ca464377bb72a14959d141de'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.175330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c875d37ca464377bb72a14959d141de'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.189740 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c875d37ca464377bb72a14959d141de'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.199625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1351/2694] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:48 2022] DELETE /identity/v3/users/1c875d37ca464377bb72a14959d141de => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:48.205473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:48.206108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:48.206375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:48.228266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:48.229611 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:48.269123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=100 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:48.275290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:48.277912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:48.312542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1344/2695] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:48.324751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:48.326358 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:48.357703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:48.388102 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d12717-e768-4308-9a24-b41089d3c549 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:48.395091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:48.396027 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:48.396537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:48.396850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] PATH_INFO: `/v3/users/cd30eb171a8040b4be937d568f0a4be2` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:48.398734 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd30eb171a8040b4be937d568f0a4be2)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:48.402156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:48.461135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd30eb171a8040b4be937d568f0a4be2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.464017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328d276-5f2b-4fe0-998e-cc6568e31dc5 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:48.470664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:48.472399 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:48.472987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:48.473561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:48.474962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd30eb171a8040b4be937d568f0a4be2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.476269 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd30eb171a8040b4be937d568f0a4be2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.495852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:48.495852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:48.502343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd30eb171a8040b4be937d568f0a4be2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.513933 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1345/2696] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:23:48.520686 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd30eb171a8040b4be937d568f0a4be2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:48.534679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1352/2697] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:23:48 2022] DELETE /identity/v3/users/cd30eb171a8040b4be937d568f0a4be2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:23:48.947182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:48.949920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:48.977717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb768a2-06ee-4305-98e2-71989f9936ea service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:48.984724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:48.986867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.060263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.061192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.061551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.061894 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] PATH_INFO: `/v3/projects/6f6de2c24f1d409889d26b51329ba089` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.063348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6de2c24f1d409889d26b51329ba089)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.065452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.086867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fdb5e-b2f0-416d-995c-28a8a9d263f9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.092205 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.093100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.093463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.093717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] PATH_INFO: `/v3/projects/53374ad6bfce4d7a9d51448ad1d4f510` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.094958 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] RBAC: Authorizing `identity:delete_project(project_id=53374ad6bfce4d7a9d51448ad1d4f510)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.097022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.119024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6de2c24f1d409889d26b51329ba089', 'request_id': 'req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.123159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:49.130737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6de2c24f1d409889d26b51329ba089', 'request_id': 'req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.139407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6de2c24f1d409889d26b51329ba089', 'request_id': 'req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.154348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53374ad6bfce4d7a9d51448ad1d4f510', 'request_id': 'req-c6977a33-ed86-445c-840c-16391a4341be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.162363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53374ad6bfce4d7a9d51448ad1d4f510', 'request_id': 'req-c6977a33-ed86-445c-840c-16391a4341be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.164484 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] The token cache is being invalidate because project 6f6de2c24f1d409889d26b51329ba089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:49.164829 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.165486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7939629e-27f6-4f2a-9362-21b7dc8c4a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.167057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1346/2698] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:48 2022] DELETE /identity/v3/projects/6f6de2c24f1d409889d26b51329ba089 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:49.168560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53374ad6bfce4d7a9d51448ad1d4f510', 'request_id': 'req-c6977a33-ed86-445c-840c-16391a4341be'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.172542 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4832cb-8fb1-4071-95fa-1e01497beec1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.172854 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4832cb-8fb1-4071-95fa-1e01497beec1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.173305 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4832cb-8fb1-4071-95fa-1e01497beec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.173682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:49.191237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] The token cache is being invalidate because project 53374ad6bfce4d7a9d51448ad1d4f510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:23:49.191962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.192633 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-c6977a33-ed86-445c-840c-16391a4341be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:23:49.193690 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-dc4832cb-8fb1-4071-95fa-1e01497beec1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.194418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1353/2699] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:23:48 2022] DELETE /identity/v3/projects/53374ad6bfce4d7a9d51448ad1d4f510 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:49.194666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dc4832cb-8fb1-4071-95fa-1e01497beec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.199456 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f635eae-7d4e-4066-844c-2d10c175ac93 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.199876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f635eae-7d4e-4066-844c-2d10c175ac93 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.200606 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f635eae-7d4e-4066-844c-2d10c175ac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.224143 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-9f635eae-7d4e-4066-844c-2d10c175ac93 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.225386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9f635eae-7d4e-4066-844c-2d10c175ac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.268495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1347/2700] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:49.276194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.276480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.276757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.292966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1354/2701] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:49.295309 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.296558 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.300973 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.301263 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.301537 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.305734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1348/2702] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:49.310168 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.320017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a2001-3d23-4f97-9ba1-3db20c3be1f7 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.320933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.322206 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.324760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.325467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.325752 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.326030 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.326751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:49.327083 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.328821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.330748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1355/2703] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:49.334971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.341816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1349/2704] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:23:49.345200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfe48d-96b5-49fd-b911-32c5c86837ae None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.345977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.350209 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.350843 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.351128 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.351398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.352171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:23:49.352478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.354097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.357552 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca20d-821c-4d2d-93db-288d9a616b0c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.362738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.363396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.363715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.364138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.364858 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.366800 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.373570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1356/2705] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 12:23:49.374744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.386176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e878904-1e13-437f-a7a2-7a09455bc0aa admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.391204 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.391908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.392183 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.392470 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.393213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.395032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1350/2706] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:23:49.395248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.399210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.426709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1357/2707] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:23:49.430976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.473574 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2df0-8e47-4a80-aae3-f1a971cbdec7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.504553 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055082e4-392a-4ea2-9b6d-e1e8a9e7d560 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.509791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.510484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.510850 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.511045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.511794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.513768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.525066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:49.539852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1358/2708] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:23:49.543550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.544238 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.544519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.544816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.545095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.547010 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.549307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.555061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3300be2-0bc7-4a94-a48c-721420a83121 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.564637 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.565702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.566189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.566567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.567754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.570462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.576719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1351/2709] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:23:49.582000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.587363 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1359/2710] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:49.591869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.593496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf76f9-5772-42a6-b3c5-9b95d7911e6f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.599345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.600203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.600715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.601164 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.602060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.603088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f4ec6-e4e2-4de3-971c-b46024ba92ac admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.604092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.608766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.609457 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.609742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.610019 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.610761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.612566 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.618682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1352/2711] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:49.623033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.631748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1360/2712] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:49.634675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.635737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252afe4f-3049-4b7e-be7a-52c8162a3a41 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.641662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.642376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.642683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.642939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.643966 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.646407 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.650604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637f610-3788-4239-b516-48af8e0d8fa9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.657188 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.658295 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.658709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.659155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1353/2713] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:23:49.662278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] PATH_INFO: `/v3/projects/bced30c3a05f4152b0b7cb5a62fd884b/users/1763238b78e549eba7728f73f4093c37/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.663116 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.676570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3efc-f1a0-4c5f-96cc-16841c8e65d0 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.682958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.683948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bced30c3a05f4152b0b7cb5a62fd884b, user_id=1763238b78e549eba7728f73f4093c37, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.684733 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.684961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.685411 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] PATH_INFO: `/v3/projects/dea45f24c1da4e3ea816610296bc6df5/users/92874c9e7ce944089f627f7074e40dae/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.688042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf7299-8e0f-4f1c-842e-94a652978be0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.701115 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:49.707629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea45f24c1da4e3ea816610296bc6df5, user_id=92874c9e7ce944089f627f7074e40dae, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.710903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1361/2714] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:49 2022] PUT /identity/v3/projects/bced30c3a05f4152b0b7cb5a62fd884b/users/1763238b78e549eba7728f73f4093c37/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:49.713514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20234cf-ac77-4117-860b-29c1e4ef7fd5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.722529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.722917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.723159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.738498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1354/2715] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:23:49 2022] PUT /identity/v3/projects/dea45f24c1da4e3ea816610296bc6df5/users/92874c9e7ce944089f627f7074e40dae/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:23:49.745231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.745891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.746491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.755595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] MFA Rules not processed for user `1763238b78e549eba7728f73f4093c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.756575 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.774503 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:49.776039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.845264 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1362/2716] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:49.847707 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.849580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.854993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1355/2717] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:23:49.869107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:49.872815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:49.937765 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c24ecf-183f-473f-ba18-a973d64ffd29 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.942410 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67243b69-4137-41ed-a86e-c35bb4105c37 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:49.942635 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.943377 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.943788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.944070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.947325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:49.949040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:49.949040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:49.949040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:49.957652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.959139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9336a-56b5-4727-a4c4-eb183a24dbf0 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.960180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:49.961713 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:49.970532 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1356/2718] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:49.972954 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1363/2719] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:23:50.716254 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6985-2acc-4af5-9dbe-47a5deb065cc None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:50.716825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6985-2acc-4af5-9dbe-47a5deb065cc None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:50.716825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6985-2acc-4af5-9dbe-47a5deb065cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:50.745069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-08bb6985-2acc-4af5-9dbe-47a5deb065cc None None] MFA Rules not processed for user `92874c9e7ce944089f627f7074e40dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:50.746342 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-08bb6985-2acc-4af5-9dbe-47a5deb065cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:50.771973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:23:50.833791 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1357/2720] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:50.850112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:50.865010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77579-951c-4e93-a0d4-4a2ad27c96af service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:50.871965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:50.873365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:50.873992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:50.874598 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:50.892383 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:50.895083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:50.909062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1364/2721] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:23:52.290405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:52.290976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:52.291101 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:52.310386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] MFA Rules not processed for user `92874c9e7ce944089f627f7074e40dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:23:52.311684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:52.320510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1358/2722] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:23:52.576717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:23:52.587711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649ee9-c6b7-4213-b86a-a599f05cd850 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:23:52.593760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:23:52.595727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:23:52.596727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:23:52.597016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:23:52.600947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:23:52.612165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:23:52.613673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c7324-4eb7-40ad-98a5-545459f97fce service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:23:52.623253 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1365/2723] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:02.303878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=19 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:02.618987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=71 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:07.649405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:07.661689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f8b4f-27ea-4300-a966-b81c50b29c88 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:07.668900 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:07.670419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:07.670889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:07.671373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] PATH_INFO: `/v3/users/1763238b78e549eba7728f73f4093c37` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:07.672885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1763238b78e549eba7728f73f4093c37)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:07.676595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:07.742018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1763238b78e549eba7728f73f4093c37'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:07.752888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1763238b78e549eba7728f73f4093c37'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:07.772312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1763238b78e549eba7728f73f4093c37'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:07.773529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1763238b78e549eba7728f73f4093c37'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:07.783951 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1763238b78e549eba7728f73f4093c37'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:07.791571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1359/2724] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:24:07 2022] DELETE /identity/v3/users/1763238b78e549eba7728f73f4093c37 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:24:07.796052 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:07.796493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:07.796600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:07.819675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:07.821509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:07.894101 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1366/2725] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:07.908364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:07.910167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:07.976033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f393c-f88e-488f-9ac3-f5bbca7756cb admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:07.980488 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:07.981177 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:07.981465 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:07.981796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:07.993277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:07.995349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389f3b0-df19-4912-adce-fd4a9458be7f service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.007192 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1360/2726] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:08.319211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.321294 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:08.338585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240987d-f6b2-47a4-8f7d-51ed4844f146 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.344588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.345478 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.345755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.346023 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] PATH_INFO: `/v3/projects/bced30c3a05f4152b0b7cb5a62fd884b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.347685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bced30c3a05f4152b0b7cb5a62fd884b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.350670 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.413461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bced30c3a05f4152b0b7cb5a62fd884b', 'request_id': 'req-e64f76b8-e59a-499d-acea-97a10b3267f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:08.424871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bced30c3a05f4152b0b7cb5a62fd884b', 'request_id': 'req-e64f76b8-e59a-499d-acea-97a10b3267f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:08.435304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bced30c3a05f4152b0b7cb5a62fd884b', 'request_id': 'req-e64f76b8-e59a-499d-acea-97a10b3267f2'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:08.450362 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] The token cache is being invalidate because project bced30c3a05f4152b0b7cb5a62fd884b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:24:08.450666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:08.451391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e64f76b8-e59a-499d-acea-97a10b3267f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:24:08.453100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1367/2727] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:24:08 2022] DELETE /identity/v3/projects/bced30c3a05f4152b0b7cb5a62fd884b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:24:08.458201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.458497 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.458775 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.479289 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:08.480707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:08.559140 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1361/2728] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:08.572259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.572578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.572844 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.592968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:08.594214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:08.603044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1368/2729] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:08.607827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.618096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde1e72-79ac-41f9-8ec3-36887bb2c458 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.622750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.623460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.623878 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.624153 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.624924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:24:08.625232 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.626831 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.640120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1362/2730] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:24:08.644842 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.655702 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c6ab1-c6df-4d46-901f-ce2a9199be69 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.662144 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.662841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.663232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.663548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.664299 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.666254 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.697677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1369/2731] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:24:08 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:24:08.702435 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.708396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:08.715233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7f616-598d-46f8-82a1-88cfdb427706 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.780157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.781360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.781888 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.782485 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.783828 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.787069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.810772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1363/2732] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:24:08 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:24:08.815045 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.826007 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd1336-481e-48d0-8c2b-0f0c7af62d83 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.831165 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.831919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.832185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.832455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.833197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.834858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.845787 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1370/2733] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:24:08.850038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.860759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e25d-05bb-48ed-9e6f-70922af9baa3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.866970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.867652 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.867934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.868207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.868920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.870541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4133cb-ca6a-4baf-81bd-03fc16dc7f0d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.882108 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1364/2734] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:24:08.886955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:08.897994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fa717-3028-442a-ba65-5900c30b4c17 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:08.902887 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:08.903855 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.904159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.904438 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] PATH_INFO: `/v3/projects/d8b94e9bdc2149bcb3b9a0f89729b257/users/bdd46019b7174470b969f25c1b708f97/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.921360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b94e9bdc2149bcb3b9a0f89729b257, user_id=bdd46019b7174470b969f25c1b708f97, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:08.924198 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:08.939679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1371/2735] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:24:08 2022] PUT /identity/v3/projects/d8b94e9bdc2149bcb3b9a0f89729b257/users/bdd46019b7174470b969f25c1b708f97/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:24:08.945096 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702589-9223-4c36-bf05-a6d16beda188 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:08.945387 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702589-9223-4c36-bf05-a6d16beda188 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:08.945699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef702589-9223-4c36-bf05-a6d16beda188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:08.965044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-ef702589-9223-4c36-bf05-a6d16beda188 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:08.966286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ef702589-9223-4c36-bf05-a6d16beda188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:09.033872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1365/2736] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:09.043721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:09.045839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:09.076756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:09.126017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf2476-266a-439f-bbfc-dd6dd6aaf74b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:09.130005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:09.130675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:09.131040 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:09.131446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:09.144122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:09.145604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:09.159173 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1372/2737] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:13.714189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:13.714903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:13.714903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:13.745774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] MFA Rules not processed for user `bdd46019b7174470b969f25c1b708f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:13.747049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:13.841895 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1366/2738] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:13.846875 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:13.857908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62178-6c07-471a-9621-f8576a8fceec service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:13.862861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:13.863563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:13.863849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:13.864129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:13.864999 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:13.867070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:13.907693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1373/2739] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:24:13 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:24:13.913526 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:13.925218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c59f3-80d6-4135-b89c-1c4e013de390 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:13.939760 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:14.043963 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:14.044577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:14.044961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:14.045336 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:14.046154 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:14.048266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:14.086577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1367/2740] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:24:13 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:24:14.092818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:14.105050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2613c-4600-47db-bc8f-d42f7d3fda5e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:14.111211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:14.111963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:14.112399 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:14.112625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:14.113270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:14.115010 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:14.128323 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1374/2741] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:24:14.133544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:14.147557 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150be57-fedd-4837-bac5-ea7ad5bf4afa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:14.162685 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:14.163157 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:14.163464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:14.163759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] PATH_INFO: `/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82/users/0db50beeb8a5427d89424f9831442d2b/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:14.183595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f03bcc64d5492f8bc27a4178d1cb82, user_id=0db50beeb8a5427d89424f9831442d2b, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:14.186838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:14.203885 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1368/2742] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:24:14 2022] PUT /identity/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82/users/0db50beeb8a5427d89424f9831442d2b/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:24:14.208900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:14.229069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631669b-7980-4984-8380-77cac3dd804c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:14.305772 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:14.306469 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:14.306773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:14.307030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:14.308186 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:14.309849 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c3add-31e4-4e84-a095-ef2e18cc71e6 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:14.320995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1375/2743] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:24:14.325759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:14.337565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d511bc-b836-444c-b85f-7f1324671762 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:14.343225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:14.344390 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:14.344746 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:14.345094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] PATH_INFO: `/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82/users/0db50beeb8a5427d89424f9831442d2b/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:14.347457 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f03bcc64d5492f8bc27a4178d1cb82, user_id=0db50beeb8a5427d89424f9831442d2b, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:14.350316 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:14.362471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1369/2744] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:24:14 2022] PUT /identity/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82/users/0db50beeb8a5427d89424f9831442d2b/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:24:20.120415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38644ca-bb86-4e0d-99b1-b8b2a54b8a5c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:20.121035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38644ca-bb86-4e0d-99b1-b8b2a54b8a5c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:20.121309 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38644ca-bb86-4e0d-99b1-b8b2a54b8a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:20.198331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c38644ca-bb86-4e0d-99b1-b8b2a54b8a5c None None] MFA Rules not processed for user `0db50beeb8a5427d89424f9831442d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:20.200924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c38644ca-bb86-4e0d-99b1-b8b2a54b8a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:20.324259 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1376/2745] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:20.338556 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:20.340886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:20.419763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a8453-0b0c-4cc4-b2c8-bb4d974ae37b admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:20.428011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f54544af7d84443e9b8fdbcb718bf824', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:20.429286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:20.429743 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:20.430095 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:20.521970 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:20.523458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:20.533665 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1370/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:20.660151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b832e1-7cf6-437d-97a3-1cdc582dbab8 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:20.660448 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b832e1-7cf6-437d-97a3-1cdc582dbab8 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:20.660731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b832e1-7cf6-437d-97a3-1cdc582dbab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:20.680612 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f2b832e1-7cf6-437d-97a3-1cdc582dbab8 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:20.682016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f2b832e1-7cf6-437d-97a3-1cdc582dbab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:20.694756 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:20.757149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1377/2747] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:20.782818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:20.789658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:20.815725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:20.875821 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef375-5bbc-49e7-b356-a471254fa861 service glance] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:20.882026 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:20.882915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:20.883262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:20.883631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:20.893741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:20.895727 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:20.905453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1371/2748] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:24:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:24:26.295185 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:26.296129 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:26.296625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:26.336555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] MFA Rules not processed for user `9341552f54ff45d8b98b54522ba9e3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:26.345193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:26.420751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1378/2749] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:29.328414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:29.330036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:29.446967 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2c787-7c01-4e92-86a3-fe8cb5e5fdfb service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:29.454114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:29.455267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:29.455808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:29.456315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:29.460352 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:29.477577 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:29.479070 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139dc154-c7ff-4df2-aae1-9b4543e04295 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:29.499604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1372/2750] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:29.508703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:29.510934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:29.709915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e2a06-49b0-4439-b8a0-58be4ca3fdcd None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:29.719845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:29.720582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:29.720919 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:29.721332 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:29.739858 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:29.742625 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c550a-6316-4374-b646-c837a172db73 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:29.762397 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1379/2751] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:24:31.934693 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:31.935614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:31.936094 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:31.967778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] MFA Rules not processed for user `bdd46019b7174470b969f25c1b708f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:31.972218 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:31.992482 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1373/2752] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:34.465868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:34.466400 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:34.466574 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:34.487344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:24:34.488711 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:24:34.496811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1380/2753] 10.222.0.18 () {58 vars in 1009 bytes} [Thu Jul 21 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:24:34.511344 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:24:34.525222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ef710-9ac7-492d-b03e-2f00cdde8581 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:24:34.534973 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:24:34.535701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:24:34.535978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:24:34.536398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:24:34.546049 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:24:34.547922 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:24:34.557923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1374/2754] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:24:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:24:44.482529 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=83 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:24:44.553416 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=59 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:25:40.936804 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:25:40.951667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f4680-2c52-4620-9cd1-02bcd2293fc5 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:25:40.958731 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:25:40.959710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:25:40.959993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:25:40.960285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:25:40.962373 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:25:41.052062 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:25:41.053866 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:25:41.065947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1381/2755] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:25:42.263203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:25:42.275835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc6495-1435-40b1-b90a-6d85585c925d service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:25:42.280187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:25:42.280835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:25:42.281119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:25:42.281408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:25:42.283398 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:25:42.297871 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:25:42.299312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:25:42.311024 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1375/2756] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:25:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:25:51.060501 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=34 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:25:52.305555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=6 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:26:33.938893 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:26:33.947471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7cfae-88a4-4ec4-904b-4e1813be0f78 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:26:33.952738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:26:33.953833 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:26:33.954274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:26:33.954708 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:26:33.956767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:26:33.968429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:26:33.969897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:26:33.980908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1382/2757] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:26:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:26:43.977203 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:28:46.423931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:28:46.425480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:28:46.500658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debccae-d8cc-4dda-a882-98792d883247 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:28:46.505956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:28:46.506730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:28:46.507017 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:28:46.507296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:28:46.509273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:28:46.522039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:28:46.523692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:28:46.534670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1376/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:28:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:28:56.530195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=32 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:29:29.678009 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:29:29.681048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:29:29.705390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9c1fe-22d0-4bfa-9039-8d0287fbeb9f service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:29:29.712286 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:29:29.712932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:29:29.713213 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:29:29.713482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:29:29.722732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:29:29.724344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:29:29.752869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1383/2759] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:29:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:29:39.749663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=13 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:30:32.938634 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:30:32.949891 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795aaa2-5a22-4a24-ac57-6c78fe075fa3 service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:30:32.954258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:30:32.955292 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:30:32.955569 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:30:32.955875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:30:32.957934 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:30:33.027809 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:30:33.029285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:30:33.039813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1377/2760] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:30:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:30:33.091824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:30:33.101483 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20db64-cbe1-4e37-82bb-2315c34d79d2 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:30:33.105145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:30:33.105793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:30:33.106074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:30:33.106348 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:30:33.115155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:30:33.116644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:30:33.126480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1384/2761] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:30:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:30:33.173932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:30:33.183838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aff2-492e-4202-8cda-056ad444a227 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:30:33.188886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:30:33.189564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:30:33.189852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:30:33.190131 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:30:33.192269 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:30:33.254811 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:30:33.256332 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:30:33.266212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1378/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:30:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:30:41.836055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:30:41.847096 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72cd8-e9dd-4b6a-aa71-a9bb07990ebd service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:30:41.852044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:30:41.852704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:30:41.853084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:30:41.853367 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:30:41.864918 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:30:41.866376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:30:41.877802 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1385/2763] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:30:42.993304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:30:43.003849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7af50-2827-483c-9f4b-95345dabf1e6 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:30:43.008238 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:30:43.009003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:30:43.009403 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:30:43.009779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:30:43.022886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:30:43.024531 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:30:43.038670 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1379/2764] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:30:51.872754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=12 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:30:53.031450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=62 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:31:49.065614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:31:49.075841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924adeef-977b-42e7-b0b3-dfa3f00acf7b service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:31:49.080347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:31:49.080981 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:31:49.081260 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:31:49.081534 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:31:49.093127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:31:49.095311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:31:49.106083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1386/2765] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:31:59.101714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:33:49.464806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:33:49.466452 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:33:49.560291 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66538cd-a240-41e6-beb4-71d5d83c42a6 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:33:49.566009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:33:49.566657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:33:49.566940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:33:49.567207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:33:49.576040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:33:49.577628 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:33:49.587637 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1380/2766] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:33:59.583751 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=37 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:34:33.747992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:34:33.750248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:34:33.854547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd3f8-cf7c-4e68-a6c9-22cdb5d1b6de service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:34:33.858150 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:34:33.858784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:34:33.859058 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:34:33.859341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:34:33.861246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:34:33.933957 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:34:33.935365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:34:33.944738 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1387/2767] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:34:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:34:43.940749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=67 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:35:32.427385 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:35:32.429917 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:35:32.481187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60233a2-47d5-4669-b40d-ac8fd49b30dd service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:35:32.487867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:35:32.488522 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:35:32.489458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:35:32.489767 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:35:32.498091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:35:32.499856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:35:32.527607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1381/2768] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:35:33.826664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:35:33.847955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9d3-7c06-414f-ac15-888721c00940 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:35:33.855452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:35:33.856130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:35:33.856415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:35:33.856693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:35:33.867695 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:35:33.869289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:35:33.897385 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1388/2769] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:35:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:35:42.525011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=25 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:35:43.894765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=13 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:36:34.196438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:36:34.206725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfbdaa-bf53-4895-947c-c3fd6c086b62 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:36:34.210850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:36:34.211541 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:36:34.211853 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:36:34.212093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:36:34.214151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:36:34.286408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:36:34.287915 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:36:34.300209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1382/2770] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:36:34.346002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:36:34.354298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925d52f-c7fd-4639-b10c-d3f3267feacc service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:36:34.358133 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:36:34.358784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:36:34.359078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:36:34.359330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:36:34.361416 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:36:34.448505 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:36:34.449961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:36:34.469515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1389/2771] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:36:34.552433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:36:34.562073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b6c6-b8a5-4009-90de-f7c817f16f51 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:36:34.566663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:36:34.567307 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:36:34.567627 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:36:34.567903 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:36:34.569942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:36:34.583754 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:36:34.585194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:36:34.596375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1383/2772] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:36:44.465622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=41 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:36:44.591642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=40 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:36:56.052197 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:36:56.053903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:36:56.137358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c5a4-c939-42c5-942b-a6caa24d498a service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:36:56.141275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:36:56.141923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:36:56.142201 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:36:56.142476 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:36:56.144725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:36:56.160237 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:36:56.161704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfa319-11bd-4405-98f1-0721a196bf36 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:36:56.170539 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1390/2773] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:36:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:37:06.167048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=35 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:38:39.219939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:39.231785 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240aec8-d4ce-4049-bcbd-b2766f6aeb9c service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:39.237354 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:39.238005 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:39.238284 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:39.238562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:39.240563 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:39.253570 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:39.255156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:39.266193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1384/2774] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:38:40.517902 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:40.518411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:40.518608 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:40.582056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] MFA Rules not processed for user `7da32d4f2caf4f0aa22f839c4a3e6b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:38:40.583310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:40.659106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1391/2775] 10.222.0.18 () {58 vars in 967 bytes} [Thu Jul 21 12:38:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:38:40.705769 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:40.707429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:40.812210 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59e8a3-631f-46f2-a447-bfbb154e3612 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:40.818514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd14dd264bd4343b49ef968789de08f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:40.819176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:40.819469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:40.819776 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:40.828523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:40.830118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:40.838530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1385/2776] 10.222.0.18 () {60 vars in 1384 bytes} [Thu Jul 21 12:38:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 12:38:41.228691 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:41.230354 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:41.243874 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f2794-ce01-4a23-bd99-b49e8ba3927a None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:41.248682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:41.249339 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:41.249622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:41.249892 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:41.251915 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:41.353636 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:41.355118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:41.373741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1392/2777] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:38:44.191883 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:44.204220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef4d5-8376-4dc5-97a2-4070be5d1b2f service swift] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:44.210810 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:44.211599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:44.211932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:44.212458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:44.214444 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:44.227850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:44.230320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:44.242509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1386/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:38:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:38:47.479121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:47.481325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:47.496281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ed97-1ccc-420f-a49a-93374ec380b8 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:47.501099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:47.502067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:47.502461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:47.502803 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] PATH_INFO: `/v3/users/bdd46019b7174470b969f25c1b708f97` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:47.504227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd46019b7174470b969f25c1b708f97)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:47.506226 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:47.532764 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:38:47.563717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd46019b7174470b969f25c1b708f97'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.570775 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd46019b7174470b969f25c1b708f97'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.571450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd46019b7174470b969f25c1b708f97'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.583063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd46019b7174470b969f25c1b708f97'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.590900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd46019b7174470b969f25c1b708f97'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.597044 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1393/2779] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:38:47 2022] DELETE /identity/v3/users/bdd46019b7174470b969f25c1b708f97 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:38:47.792610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:47.817357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d748cd1-90e2-441e-9ee9-935d410af75a service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:47.874939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:47.875843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:47.876123 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:47.876395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] PATH_INFO: `/v3/projects/d8b94e9bdc2149bcb3b9a0f89729b257` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:47.877616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b94e9bdc2149bcb3b9a0f89729b257)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:47.879588 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:47.932929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:38:47.936331 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b94e9bdc2149bcb3b9a0f89729b257', 'request_id': 'req-293cc22b-93a5-4593-b268-5891cb3329ea'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.946058 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b94e9bdc2149bcb3b9a0f89729b257', 'request_id': 'req-293cc22b-93a5-4593-b268-5891cb3329ea'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.952586 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b94e9bdc2149bcb3b9a0f89729b257', 'request_id': 'req-293cc22b-93a5-4593-b268-5891cb3329ea'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.964329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] The token cache is being invalidate because project d8b94e9bdc2149bcb3b9a0f89729b257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:38:47.964704 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.965567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:47.967914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1387/2780] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:38:47 2022] DELETE /identity/v3/projects/d8b94e9bdc2149bcb3b9a0f89729b257 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:38:47.971911 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:47.973395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:47.986547 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379a8a-b0e7-40bf-9dbc-201f74c8b28a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:48.050180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:48.051176 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:48.051644 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:48.052031 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] PATH_INFO: `/v3/users/0db50beeb8a5427d89424f9831442d2b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:48.066747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db50beeb8a5427d89424f9831442d2b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:48.068795 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:48.127679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db50beeb8a5427d89424f9831442d2b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.135080 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db50beeb8a5427d89424f9831442d2b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.135869 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db50beeb8a5427d89424f9831442d2b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.149242 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db50beeb8a5427d89424f9831442d2b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.159861 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db50beeb8a5427d89424f9831442d2b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.166025 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1394/2781] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:38:47 2022] DELETE /identity/v3/users/0db50beeb8a5427d89424f9831442d2b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:38:48.351458 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:48.353109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:48.366523 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc22b-93a5-4593-b268-5891cb3329ea admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:48.433832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:48.434717 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:48.434992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:48.435265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] PATH_INFO: `/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:48.445940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f03bcc64d5492f8bc27a4178d1cb82)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:48.447997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:48.499151 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f03bcc64d5492f8bc27a4178d1cb82', 'request_id': 'req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.509505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f03bcc64d5492f8bc27a4178d1cb82', 'request_id': 'req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.516937 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f03bcc64d5492f8bc27a4178d1cb82', 'request_id': 'req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.527728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] The token cache is being invalidate because project 64f03bcc64d5492f8bc27a4178d1cb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:38:48.528014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.528683 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:38:48.530252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1388/2782] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:38:48 2022] DELETE /identity/v3/projects/64f03bcc64d5492f8bc27a4178d1cb82 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:38:56.601270 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:38:56.603986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:56.620569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:38:56.687208 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28978051-b9d8-488d-a52f-e150217f0b18 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:38:56.748662 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:38:56.749239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:38:56.749518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:38:56.749808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:38:56.751749 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:38:56.763038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:38:56.764652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:38:56.774941 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1395/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:38:58.524816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:DELETE=11,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:39:06.770434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=54 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:39:33.369066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:39:33.372008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:39:33.434244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1bd9-74fe-4af5-b637-6bfa8f915c9c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:39:33.439093 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:39:33.439784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:39:33.440066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:39:33.440363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:39:33.442394 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:39:33.511734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:39:33.513189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:39:33.531965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1389/2784] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:39:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:39:43.529318 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=56 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:40:32.786375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:40:32.797225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab7730-f664-43b3-abab-a03beedde1bf service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:40:32.801639 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:40:32.802292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:40:32.802581 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:40:32.802857 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:40:32.805000 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:40:32.879908 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:40:32.881361 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:40:32.890463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1396/2785] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:40:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:40:34.143953 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:40:34.159510 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cb7bc-3111-485d-93ea-ba6d15db605b service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:40:34.164008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:40:34.164664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:40:34.164949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:40:34.165225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:40:34.167381 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:40:34.241850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:40:34.243209 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:40:34.252445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1390/2786] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:40:42.886330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=33 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:40:44.248349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=33 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:42:01.873337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:42:01.874925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:42:01.933274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51245587-c790-45f4-b375-cf7aad2139e6 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:42:01.937555 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:42:01.938172 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:42:01.938482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:42:01.938755 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:42:01.940863 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:42:01.951012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:42:01.952491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:42:01.960947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1397/2787] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:42:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:42:11.956995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=26 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:42:34.808480 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:42:34.810519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:42:34.827201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c22a-020f-45a9-8453-7ef5aceb1b7a service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:42:34.831349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:42:34.832038 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:42:34.832329 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:42:34.832614 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:42:34.834417 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:42:34.904598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:42:34.906032 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:42:34.917260 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1391/2788] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:42:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:42:36.115236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:42:36.124912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed927f-9083-4272-a3d4-2c6328ae7903 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:42:36.129038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:42:36.129675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:42:36.129953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:42:36.130338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:42:36.132316 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:42:36.146580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:42:36.148191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:42:36.161157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1398/2789] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:42:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:42:44.912616 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=34 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:42:46.156609 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:44:03.419011 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:44:03.420540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:03.500349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38592c-5c16-4a21-9979-5e8dabe95698 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:44:03.506773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:44:03.507426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:44:03.507753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:44:03.508040 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:44:03.517267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:44:03.518864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:44:03.529413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1392/2790] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:44:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:44:13.524805 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=32 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:44:34.138809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:44:34.140418 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:34.206622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea439186-8cef-42ac-b541-d06eb7f87582 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:44:34.210797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:44:34.211567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:44:34.211765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:44:34.212053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:44:34.222240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:44:34.223827 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:44:34.237427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1399/2791] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:44:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:44:39.328740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:44:39.333277 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:39.361562 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c90461-8ce6-42c2-97cb-e5fc091199bf service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:44:39.367613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:44:39.368841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:44:39.369194 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:44:39.369540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:44:39.371772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:39.398657 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:44:39.400356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:44:39.423911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1393/2792] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:44:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:44:40.448012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:44:40.449736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:40.472042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75168e-e0a6-43b0-884d-0f98c46c21db service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:44:40.478170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:44:40.479053 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:44:40.479538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:44:40.479991 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:44:40.482819 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:44:40.497403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:44:40.499275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:44:40.517603 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1400/2793] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:44:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:44:49.408330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=11 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:44:50.506641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=34 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:45:33.239244 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:45:33.268906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c8f0d-15d4-456e-a9f9-b342c05ef1e4 service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:45:33.276281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:45:33.276924 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:45:33.277195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:45:33.277461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:45:33.286914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:45:33.288593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:45:33.317797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1394/2794] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:45:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:45:35.397098 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:45:35.434968 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49250ab7-7e53-4c55-9e13-ce5b44782749 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:45:35.441463 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:45:35.442335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:45:35.442480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:45:35.442663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:45:35.451265 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:45:35.453008 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:45:35.481083 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1401/2795] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:45:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:45:43.314699 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=20 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:45:45.477971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=20 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:47:34.982327 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:47:35.002348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b138eaa-a888-4e4f-b523-fc3f34e76c0d service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:47:35.006780 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:47:35.007432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:47:35.007739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:47:35.008012 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:47:35.019280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:47:35.020747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:47:35.031761 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1395/2796] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:47:36.141668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:47:36.154585 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9008965-44a1-4aa3-b5e6-979af438cbc2 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:47:36.159277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:47:36.159962 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:47:36.160248 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:47:36.160524 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:47:36.172491 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:47:36.173947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:47:36.185338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1402/2797] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:47:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:47:45.026994 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=10 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:47:46.180106 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=6 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:49:07.399355 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:49:07.401127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:49:07.483112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0c8bd-487c-4a6a-b3a7-5f730411ce21 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:49:07.488730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:49:07.489379 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:49:07.489660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:49:07.489935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:49:07.492003 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:49:07.504965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:49:07.506571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:49:07.517159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1396/2798] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:49:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:49:17.512629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=32 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:49:34.490423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:49:34.502415 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561871d6-c356-4f5d-abaf-90a9d6d8e457 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:49:34.509614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:49:34.510255 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:49:34.510535 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:49:34.510807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:49:34.512953 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:49:34.606768 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:49:34.608441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:49:34.627347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1403/2799] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:49:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:49:44.625231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=41 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:50:32.707124 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:32.727069 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df685c-b003-4690-b013-ab46c4f03e1c service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:32.734193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:32.734936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:32.735196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:32.735472 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:32.745382 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:32.747008 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8020891-6ecb-45b9-9f57-5d58fde40b16 service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:32.756681 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1397/2800] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:40.680343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:40.682145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:40.741274 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:40.741914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:40.741914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:40.747370 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae16fe-0aad-4c81-b3ed-a2872ebbe5f2 service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:40.751439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:40.752119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:40.752402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:40.752959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:40.754693 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:40.782405 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] MFA Rules not processed for user `44b32f2ca6d74412b8aaf6bdc15b252a`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:50:40.783696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:40.828814 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:40.830276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:40.838957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1398/2801] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:50:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:50:40.839429 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1404/2802] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:41.151979 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:41.162676 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d097d4d-c11f-4604-b019-933ca5d276c0 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:41.166559 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:41.167203 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:41.167494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:41.167793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:41.169728 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:41.193467 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:41.194931 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:41.203118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1399/2803] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:43.589993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:43.602649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77b110-b379-4a66-8322-b788e29286d9 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:43.609130 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:43.609767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:43.610059 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:43.610328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:43.612508 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:43.624995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:43.626595 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:43.637200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1405/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:48.623598 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:48.633020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0698cab-5cb7-4026-8b77-999a78e0308e service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:48.636920 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:48.637578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:48.637861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:48.638144 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:48.647923 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:48.649651 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:48.658797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1400/2805] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:49.307742 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:49.334563 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40364e8-bccd-4577-b298-60871dbb8f3f service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:49.343329 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:49.344171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:49.344407 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:49.344820 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:49.356246 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:49.357864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:49.370002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1406/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:53.792304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:50:53.807061 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7bf7-f789-4fab-a6f8-70934cd996dc service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:50:53.813126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bccc158b75eb41ca8431bbb67af65260', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:50:53.813782 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:50:53.814067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:50:53.814346 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:50:53.816225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:50:53.843797 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:50:53.845505 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:50:53.861692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1401/2807] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:50:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:50:59.364838 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=65 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:51:03.857073 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=67 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:52:35.102966 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:52:35.105160 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:52:35.171450 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e8f4d-cb7a-4e62-86b0-7a8ce432cadb service nova] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:52:35.174977 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:52:35.175677 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:52:35.175963 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:52:35.176233 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:52:35.179111 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:52:35.272435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:52:35.273894 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:52:35.285531 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1407/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:52:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:52:36.471361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:52:36.473133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:52:36.485869 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b310-54b6-4907-ab33-4e67f69eb45c service nova] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:52:36.495686 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:52:36.496610 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:52:36.497122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:52:36.497440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:52:36.499916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:52:36.518191 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:52:36.519756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:52:36.533339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1402/2809] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:52:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:52:45.280126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=54 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:52:46.526918 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=6 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:54:13.371300 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:54:13.417026 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71ace3-6a4d-4957-803e-f8b7018a648c service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:54:13.423149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:54:13.423815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:54:13.424099 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:54:13.424372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:54:13.433315 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:54:13.434898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:54:13.445337 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1408/2810] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:54:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:54:23.441012 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=19 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:54:35.793459 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:54:35.795122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:54:35.850386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc6d29-cb87-47f2-99d2-371548f811de service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:54:35.857439 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:54:35.858345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:54:35.858749 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:54:35.859281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:54:35.869134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:54:35.870849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:54:35.880455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1403/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:54:45.876122 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=23 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:48.290492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:48.312318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333bc0-4a16-4fde-864e-1c93dcde101f service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:48.316816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:48.317482 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:48.317763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:48.318180 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:48.328687 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:48.330077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:48.351278 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:48.361212 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1409/2812] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:48.374025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c60b07-294c-4c35-8700-947efb050617 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:48.379234 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:48.379983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:48.380241 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:48.380521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:48.382498 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:48.414758 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:48.416258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:48.441958 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1404/2813] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:55:49.863077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:49.873043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b0ab-95e3-44b6-9bc5-f4ebcd35c35c service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:49.879994 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:49.880666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:49.880949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:49.881227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:49.883039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:49.984955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:49.986727 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:50.008675 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1410/2814] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:50.980881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:50.995186 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe29f5-d5bc-423c-bbf2-b349e8e7c19a service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:51.015167 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:51.016772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:51.017257 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:51.018063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:51.020633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:51.039893 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:51.042401 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:51.060898 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1405/2815] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:51.308298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:51.317793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f69fed-4a05-4864-a6e5-7394f71a120b service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:51.322142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:51.322967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:51.323413 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:51.323724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:51.326311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:51.341871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:51.343371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:51.353959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1411/2816] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:51.716720 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:51.718412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:51.750299 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30ea9b-9c83-40d0-ade7-627c6a01fff5 service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:51.755640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:51.756308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:51.756589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:51.756859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:51.758682 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:51.771815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:51.773426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:51.795645 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1406/2817] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:51.811234 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:51.813134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:51.827021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82182d4-a3a3-4fb3-92d9-98478aef5882 service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:51.833257 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:51.834421 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:51.834797 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:51.835453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] PATH_INFO: `/v3/users/92874c9e7ce944089f627f7074e40dae` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:51.836752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92874c9e7ce944089f627f7074e40dae)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:51.838969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:51.897285 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92874c9e7ce944089f627f7074e40dae'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:51.906171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92874c9e7ce944089f627f7074e40dae'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:51.907461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92874c9e7ce944089f627f7074e40dae'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:51.917635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:51.924061 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92874c9e7ce944089f627f7074e40dae'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:51.935120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92874c9e7ce944089f627f7074e40dae'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:51.942663 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1412/2818] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:55:51 2022] DELETE /identity/v3/users/92874c9e7ce944089f627f7074e40dae => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:52.371715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:52.382440 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238d7f6-44b8-4aa3-9893-bad8718323e7 service placement] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:52.446817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:52.447808 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:52.448213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:52.448589 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] PATH_INFO: `/v3/projects/dea45f24c1da4e3ea816610296bc6df5` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:52.449925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea45f24c1da4e3ea816610296bc6df5)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:52.452057 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:52.516773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea45f24c1da4e3ea816610296bc6df5', 'request_id': 'req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:52.527835 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea45f24c1da4e3ea816610296bc6df5', 'request_id': 'req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:52.537395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea45f24c1da4e3ea816610296bc6df5', 'request_id': 'req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:52.556050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] The token cache is being invalidate because project dea45f24c1da4e3ea816610296bc6df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:55:52.556345 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:52.560020 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:52.568799 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1407/2819] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:55:52 2022] DELETE /identity/v3/projects/dea45f24c1da4e3ea816610296bc6df5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:53.848570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:53.850635 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:53.882236 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160e329-a1e5-4971-94aa-1450b1adde10 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:53.949593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:53.950541 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:53.950832 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:53.951108 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] PATH_INFO: `/v3/users/44b32f2ca6d74412b8aaf6bdc15b252a` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:53.952494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b32f2ca6d74412b8aaf6bdc15b252a)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:53.954481 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.013668 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b32f2ca6d74412b8aaf6bdc15b252a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.021005 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b32f2ca6d74412b8aaf6bdc15b252a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.021876 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b32f2ca6d74412b8aaf6bdc15b252a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.034924 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b32f2ca6d74412b8aaf6bdc15b252a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.044647 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-8ea3c18d-7287-4cd8-9ac4-8fd243455aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b32f2ca6d74412b8aaf6bdc15b252a'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.054752 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1413/2820] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:55:53 2022] DELETE /identity/v3/users/44b32f2ca6d74412b8aaf6bdc15b252a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:54.239875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.241534 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:54.258475 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e56bf-464c-4dd4-b0a8-75420a39eba3 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.268262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:54.322281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.323705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.324022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.324313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] PATH_INFO: `/v3/projects/896cc13a4d744c06b80fc97aa4bd1dcf` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.325597 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896cc13a4d744c06b80fc97aa4bd1dcf)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.328133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.380208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896cc13a4d744c06b80fc97aa4bd1dcf', 'request_id': 'req-7f91de82-a682-4c38-945d-024aff035754'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.393180 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896cc13a4d744c06b80fc97aa4bd1dcf', 'request_id': 'req-7f91de82-a682-4c38-945d-024aff035754'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.402021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896cc13a4d744c06b80fc97aa4bd1dcf', 'request_id': 'req-7f91de82-a682-4c38-945d-024aff035754'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.417867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] The token cache is being invalidate because project 896cc13a4d744c06b80fc97aa4bd1dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:55:54.418356 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.418833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7f91de82-a682-4c38-945d-024aff035754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:54.420764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1408/2821] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:55:54 2022] DELETE /identity/v3/projects/896cc13a4d744c06b80fc97aa4bd1dcf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:54.427402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.427975 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.428461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.509751 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:55:54.511018 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:54.545710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:54.582684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1414/2822] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:55:54.591044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.591334 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.591658 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.611897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:55:54.613132 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:54.622060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1409/2823] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:55:54.625888 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.637158 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec8a52-1e34-497f-b684-9048b7c6c2fb None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.642345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.643030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.643327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.643653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.644395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:55:54.644701 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.646544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.665322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.667698 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1415/2824] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:55:54.676724 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76165e0-abd5-4bf3-a531-9b7a316ecd77 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.686130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.687063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.687442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.687838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.688837 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.691369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.748654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1410/2825] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:55:54 2022] POST /identity/v3/projects => generated 391 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:55:54.752825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.772506 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48f50-bf05-4a37-9662-7e14853ee6df admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.844648 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.845402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.845685 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.845961 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.846657 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.848660 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.871436 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1416/2826] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:55:54 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:55:54.875660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.886700 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8baa7fe-e5a7-4ef7-be15-48740019f16f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.892317 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.893072 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.893362 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.893640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.894396 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.896074 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.910098 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1411/2827] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:55:54.914319 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.925325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8f77-9639-4a51-951c-f18b2d195ddb admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.930355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.931808 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.932200 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.932592 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:54.933459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:54.935192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:54.949630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1417/2828] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:55:54.950756 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:54.972836 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d48e4-19a2-4d0c-8b2c-532379cb6b68 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:54.979231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:54.980722 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:54.980959 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:54.981219 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] PATH_INFO: `/v3/projects/7bf83d0293c04513b01e13b9636eac1c/users/ce2e9fc916f14b0bad643acb9f9b610b/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.001512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.002239 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf83d0293c04513b01e13b9636eac1c, user_id=ce2e9fc916f14b0bad643acb9f9b610b, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.003554 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.005349 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00d115-feac-45cc-afd4-f575e8ea9ac1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.026120 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1412/2829] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:55:54 2022] PUT /identity/v3/projects/7bf83d0293c04513b01e13b9636eac1c/users/ce2e9fc916f14b0bad643acb9f9b610b/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:55.035862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.036159 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.036443 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.099388 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] MFA Rules not processed for user `ce2e9fc916f14b0bad643acb9f9b610b`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:55:55.100992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.119704 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c06-9524-4637-883e-15a7f780427a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.139518 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:55.178640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:55.206512 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.208155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.208155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.208404 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.210570 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.223085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.224747 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.228672 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1413/2830] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:55:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:55:55.233092 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.237896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1418/2831] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:55.295991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3c79f-6188-4baa-8cbd-6c0f6094c76a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.300631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.301304 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.302220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.302506 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.303220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.305183 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.334500 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1414/2832] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:55:55 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:55:55.338432 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.349454 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055d16c-459e-4f29-91d1-521667f01841 service placement] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.407935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.408730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.409142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.409423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.410126 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.412078 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.434281 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1419/2833] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:55:55 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:55:55.438207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.448789 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f36d5-cfcf-4444-a21d-58e133fb5ca7 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.453734 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.454431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.454712 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.454987 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.455798 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.457421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.468615 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1415/2834] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:55:55.473038 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.484934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031171df-7d4b-476d-a2f5-24e42e25403f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.489561 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.490614 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.490898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.491175 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] PATH_INFO: `/v3/projects/2ccda848b67649bf83e823c338a6fa95/users/e665da6b420b492abee72e9104824b70/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.505853 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccda848b67649bf83e823c338a6fa95, user_id=e665da6b420b492abee72e9104824b70, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.508573 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.523085 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1420/2835] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:55:55 2022] PUT /identity/v3/projects/2ccda848b67649bf83e823c338a6fa95/users/e665da6b420b492abee72e9104824b70/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:55.527437 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.538715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f27e0-62b8-4fc0-8a72-960c36048a7d admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.597173 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.597861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.598139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.598414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.599233 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.600932 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dac649-fbfa-4383-ba1a-7ef077be50cf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.610742 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1416/2836] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:55:55.615091 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.625936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f17344-712c-4afc-a455-a9cb6ce8b47a admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.630971 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.632013 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.632289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.632557 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] PATH_INFO: `/v3/projects/2ccda848b67649bf83e823c338a6fa95/users/e665da6b420b492abee72e9104824b70/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.634628 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ccda848b67649bf83e823c338a6fa95, user_id=e665da6b420b492abee72e9104824b70, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.637330 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.648497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1421/2837] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:55:55 2022] PUT /identity/v3/projects/2ccda848b67649bf83e823c338a6fa95/users/e665da6b420b492abee72e9104824b70/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:55.660939 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.661230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.661509 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.687981 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] MFA Rules not processed for user `e665da6b420b492abee72e9104824b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:55:55.689271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.706408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:55.769833 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1417/2838] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:55:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:55:55.784434 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.786311 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.842730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeeeac-5681-4103-bae3-e014799e7cc9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.846667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.847317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.847623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.847895 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:55.857390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:55.858781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:55.868386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1422/2839] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:55.915235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:55.916806 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:55.932339 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838dabe-7350-4a1b-8762-b3b00aff0ca6 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:55.938268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:55.939037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:55.939524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:55.939772 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:56.004916 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:56.006730 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:56.019851 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1418/2840] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:55:58.369390 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:58.375744 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:58.407103 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:58.443043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bf0f-0d96-41f7-bed9-1215855d03f8 service cinder] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:58.447903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:58.448790 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:58.449069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:58.449346 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] PATH_INFO: `/v3/users/9341552f54ff45d8b98b54522ba9e3f9` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:58.450549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9341552f54ff45d8b98b54522ba9e3f9)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:58.452584 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:58.509003 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9341552f54ff45d8b98b54522ba9e3f9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.516435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9341552f54ff45d8b98b54522ba9e3f9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.517159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9341552f54ff45d8b98b54522ba9e3f9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.529121 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9341552f54ff45d8b98b54522ba9e3f9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.536920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9341552f54ff45d8b98b54522ba9e3f9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.543274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1423/2841] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:55:58 2022] DELETE /identity/v3/users/9341552f54ff45d8b98b54522ba9e3f9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:58.731896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:58.743583 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bddcb9-cc3c-4c86-aa59-d48aff000d34 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:58.803205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:58.804036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:58.804323 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:58.804603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] PATH_INFO: `/v3/projects/cc3651ce859043d2a764fa47f06db374` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:58.805807 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc3651ce859043d2a764fa47f06db374)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:58.807849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:58.855870 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc3651ce859043d2a764fa47f06db374', 'request_id': 'req-9724d0b3-5e75-403b-bc28-55bd917ade26'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.859550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:58.864896 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc3651ce859043d2a764fa47f06db374', 'request_id': 'req-9724d0b3-5e75-403b-bc28-55bd917ade26'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.872445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc3651ce859043d2a764fa47f06db374', 'request_id': 'req-9724d0b3-5e75-403b-bc28-55bd917ade26'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.883955 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] The token cache is being invalidate because project cc3651ce859043d2a764fa47f06db374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:55:58.884266 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.884971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:58.886629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1419/2842] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:55:58 2022] DELETE /identity/v3/projects/cc3651ce859043d2a764fa47f06db374 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:58.890746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:58.892461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:58.907298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d626f-a322-461c-ae7c-90a6144fe983 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:58.966095 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:58.967017 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:58.967298 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:58.967580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] PATH_INFO: `/v3/users/b05a208c268d44658abceddd4601d63c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:58.982452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05a208c268d44658abceddd4601d63c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:58.984509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:59.027926 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:59.038859 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05a208c268d44658abceddd4601d63c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.046391 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05a208c268d44658abceddd4601d63c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.047087 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05a208c268d44658abceddd4601d63c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.059247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05a208c268d44658abceddd4601d63c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.067142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05a208c268d44658abceddd4601d63c'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.073459 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1424/2843] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:55:58 2022] DELETE /identity/v3/users/b05a208c268d44658abceddd4601d63c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:59.263033 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:59.265060 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:59.284925 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724d0b3-5e75-403b-bc28-55bd917ade26 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:59.342629 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:59.343412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:59.343719 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:59.343991 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] PATH_INFO: `/v3/projects/b21130dbb3a84e40b87422929d0b038c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:59.355441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21130dbb3a84e40b87422929d0b038c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:59.357466 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:59.406624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21130dbb3a84e40b87422929d0b038c', 'request_id': 'req-169b9c7f-6550-49f5-b0db-b0c664c553e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.414325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21130dbb3a84e40b87422929d0b038c', 'request_id': 'req-169b9c7f-6550-49f5-b0db-b0c664c553e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.421285 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21130dbb3a84e40b87422929d0b038c', 'request_id': 'req-169b9c7f-6550-49f5-b0db-b0c664c553e0'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.432421 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] The token cache is being invalidate because project b21130dbb3a84e40b87422929d0b038c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:55:59.432707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.433337 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-169b9c7f-6550-49f5-b0db-b0c664c553e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.434932 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:59.435389 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1420/2844] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:55:59 2022] DELETE /identity/v3/projects/b21130dbb3a84e40b87422929d0b038c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:55:59.436589 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:59.450219 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cbcad-2260-4434-bfa0-64a2aba135ea admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:59.518597 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:59.519556 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:59.519841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:59.520156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] PATH_INFO: `/v3/users/ce2e9fc916f14b0bad643acb9f9b610b` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:59.521583 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2e9fc916f14b0bad643acb9f9b610b)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:59.523658 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:59.580996 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2e9fc916f14b0bad643acb9f9b610b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.588118 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2e9fc916f14b0bad643acb9f9b610b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.589050 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2e9fc916f14b0bad643acb9f9b610b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.601485 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2e9fc916f14b0bad643acb9f9b610b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.610834 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2e9fc916f14b0bad643acb9f9b610b'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:55:59.618105 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1425/2845] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:55:59 2022] DELETE /identity/v3/users/ce2e9fc916f14b0bad643acb9f9b610b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:55:59.622856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:59.623145 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:59.623445 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:59.642771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:55:59.644081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:59.713642 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1421/2846] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:55:59.723823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:55:59.725267 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:55:59.763271 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:55:59.788917 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca6864-d914-44d0-be3a-820c02a84232 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:55:59.793022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:55:59.793667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:55:59.793949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:55:59.794225 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:55:59.803748 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:55:59.805191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:55:59.814649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1426/2847] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:00.084688 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:00.087067 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:00.097486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:00.102115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e4628-6f71-4409-a19e-755dcde22873 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:00.109265 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:00.111115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:00.111788 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:00.112507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] PATH_INFO: `/v3/projects/7bf83d0293c04513b01e13b9636eac1c` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:00.113826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf83d0293c04513b01e13b9636eac1c)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:00.116393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:00.303599 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf83d0293c04513b01e13b9636eac1c', 'request_id': 'req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.311179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf83d0293c04513b01e13b9636eac1c', 'request_id': 'req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.318133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf83d0293c04513b01e13b9636eac1c', 'request_id': 'req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.329947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] The token cache is being invalidate because project 7bf83d0293c04513b01e13b9636eac1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:00.330231 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.330935 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.332550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1422/2848] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:00 2022] DELETE /identity/v3/projects/7bf83d0293c04513b01e13b9636eac1c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:00.336676 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:00.346903 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983cdfd-7c99-45a3-8169-291c56ac903d service neutron] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:00.406807 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:00.407741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:00.408022 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:00.408297 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] PATH_INFO: `/v3/users/e665da6b420b492abee72e9104824b70` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:00.409569 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e665da6b420b492abee72e9104824b70)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:00.411515 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:00.461881 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e665da6b420b492abee72e9104824b70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.468811 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e665da6b420b492abee72e9104824b70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.469503 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e665da6b420b492abee72e9104824b70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.481368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e665da6b420b492abee72e9104824b70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.489484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-e11ee482-8b45-4095-9312-3f99ab1ef52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e665da6b420b492abee72e9104824b70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.495439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1427/2849] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:00 2022] DELETE /identity/v3/users/e665da6b420b492abee72e9104824b70 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:00.777353 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:00.778876 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:00.793608 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a18eaf-ef16-45fe-bda2-7851fcfab4a5 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:00.855325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:00.856182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:00.856429 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:00.856695 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] PATH_INFO: `/v3/projects/2ccda848b67649bf83e823c338a6fa95` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:00.858163 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ccda848b67649bf83e823c338a6fa95)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:00.860121 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:00.908195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ccda848b67649bf83e823c338a6fa95', 'request_id': 'req-0f40911c-980a-4900-826e-ea245b13e009'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.917818 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ccda848b67649bf83e823c338a6fa95', 'request_id': 'req-0f40911c-980a-4900-826e-ea245b13e009'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.923957 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ccda848b67649bf83e823c338a6fa95', 'request_id': 'req-0f40911c-980a-4900-826e-ea245b13e009'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.934134 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] The token cache is being invalidate because project 2ccda848b67649bf83e823c338a6fa95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80383) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:00.934414 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.935068 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-0f40911c-980a-4900-826e-ea245b13e009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:00.936607 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1423/2850] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:00 2022] DELETE /identity/v3/projects/2ccda848b67649bf83e823c338a6fa95 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:00.941860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:00.942148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:00.942435 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:00.960728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:00.961979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:01.011276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:01.025770 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1428/2851] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:56:01.033911 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.034193 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.034464 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.052942 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:01.054182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:01.063571 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1424/2852] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:01.067816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.077664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af507b-ea7d-41c1-a77c-b08ebfd58caa None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.083039 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.083725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.084004 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.084277 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] PATH_INFO: `/v3/domains` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.085001 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:56:01.085304 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.086910 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.099033 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1429/2853] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:56:01.103519 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.113530 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd5e4e-6f9b-4d05-85c5-a148ede17c11 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.118420 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.119088 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.119365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.119673 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.120364 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.122268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.130494 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:01.149969 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1425/2854] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:01.154224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.164935 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359094ee-7d43-4600-a64e-f825d3f5d14b admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.222784 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.223445 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.223760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.224035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.224725 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.226649 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.246998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1430/2855] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:01.251196 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.261793 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e84eea-ab3b-401b-be20-2d2c9c1f6a74 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.267687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.268371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.268631 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.268905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.269650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.271259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.281795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1426/2856] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:01.285817 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.296313 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912d9a8-cff5-4850-9abe-f2260adc29f3 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.301447 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.302074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.302356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.302627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.303375 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.304992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3461d2-e92f-4463-9b71-d759a1793242 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.315386 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1431/2857] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:01.319654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.330250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab44c3b-9628-4936-aaab-336384eaaf77 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.336160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.337179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.337461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.337738 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] PATH_INFO: `/v3/projects/0af26894d1464b9bbb6a076952286374/users/81f00da5c8eb4bf5aeaa11d50f83af0a/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.352949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af26894d1464b9bbb6a076952286374, user_id=81f00da5c8eb4bf5aeaa11d50f83af0a, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.355664 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.370971 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1427/2858] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:01 2022] PUT /identity/v3/projects/0af26894d1464b9bbb6a076952286374/users/81f00da5c8eb4bf5aeaa11d50f83af0a/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:01.382765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.383060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.383338 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.411063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] MFA Rules not processed for user `81f00da5c8eb4bf5aeaa11d50f83af0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:01.412318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:01.482220 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1432/2859] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:01.486803 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.550784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b45fb-ec89-47db-9001-38b629b3963c admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.556044 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.556692 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.556975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.557249 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.557947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.559901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.588524 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1428/2860] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:01.593137 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.605824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a2f64-681d-4e0f-8a19-8b922f48b201 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.655898 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:01.666735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.667408 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.667717 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.667992 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] PATH_INFO: `/v3/users` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.668681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.670643 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.692743 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1433/2861] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:56:01.696839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.707045 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835685-bec3-463e-8714-4e9926817488 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.712887 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.713540 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.713813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.714081 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.714838 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.716455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.727212 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1429/2862] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:56:01.731292 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.742016 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f09f5-777b-4f46-b6a5-7e778f32cdfa admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.747035 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.747949 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.748232 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.748489 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] PATH_INFO: `/v3/projects/a2dba3963fe643e8885aff33fb6a995c/users/68e8bf7bde9c4935aa175dd83ef1a71f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.763464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2dba3963fe643e8885aff33fb6a995c, user_id=68e8bf7bde9c4935aa175dd83ef1a71f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.766215 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.781730 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1434/2863] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:01 2022] PUT /identity/v3/projects/a2dba3963fe643e8885aff33fb6a995c/users/68e8bf7bde9c4935aa175dd83ef1a71f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:01.786156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.800816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061076a-ce45-4a49-9015-118736d80647 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.861859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.862529 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.862814 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.863100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.863868 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.865487 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92a64c-fb0e-43a4-877b-58dcbf044602 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.875286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1430/2864] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:01.881119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:01.895376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d83dd-4641-4b1b-9a8c-073bc3a12ede admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:01.900382 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:01.901274 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.901548 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.901816 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] PATH_INFO: `/v3/projects/a2dba3963fe643e8885aff33fb6a995c/users/68e8bf7bde9c4935aa175dd83ef1a71f/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.903934 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2dba3963fe643e8885aff33fb6a995c, user_id=68e8bf7bde9c4935aa175dd83ef1a71f, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:01.907021 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:01.918134 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1435/2865] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:01 2022] PUT /identity/v3/projects/a2dba3963fe643e8885aff33fb6a995c/users/68e8bf7bde9c4935aa175dd83ef1a71f/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:01.929689 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:01.930097 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:01.930469 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:01.966600 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] MFA Rules not processed for user `68e8bf7bde9c4935aa175dd83ef1a71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:01.967863 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:01.979661 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:02.039259 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1431/2866] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:02.050689 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:02.053388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:02.120579 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f12c1a-55a3-4091-86f3-78a1cfc2d3bf admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:02.124839 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:02.125484 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:02.125763 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:02.126037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:02.135523 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:02.136951 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8cea0-0c44-482e-a304-10b38a1e6772 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:02.146538 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1436/2867] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:02.291632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:02.357208 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a307ca-d1b5-4c93-908f-96ff5031c8e1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:02.362296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:02.363172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:02.363446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:02.363750 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] PATH_INFO: `/v3/users/81f00da5c8eb4bf5aeaa11d50f83af0a` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:02.364988 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f00da5c8eb4bf5aeaa11d50f83af0a)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:02.366914 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:02.418495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f00da5c8eb4bf5aeaa11d50f83af0a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:02.427357 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f00da5c8eb4bf5aeaa11d50f83af0a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:02.440014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f00da5c8eb4bf5aeaa11d50f83af0a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:02.440653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f00da5c8eb4bf5aeaa11d50f83af0a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:02.447585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f00da5c8eb4bf5aeaa11d50f83af0a'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:02.453408 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1432/2868] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:02 2022] DELETE /identity/v3/users/81f00da5c8eb4bf5aeaa11d50f83af0a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:02.458120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:02.458409 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:02.458683 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:02.477214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:02.478411 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:02.540335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1437/2869] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:02.552968 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:02.554889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:02.588014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:02.615172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90e97-8ba2-438a-9282-7d0e378728f8 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:02.618890 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:02.619564 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:02.619841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:02.620115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:02.629126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:02.630553 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:02.639956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1433/2870] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:02.933142 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:02.934916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:02.948069 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab392c9-1572-4e85-9afa-d19207ed0e09 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:02.953182 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:02.954051 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:02.954344 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:02.954599 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] PATH_INFO: `/v3/projects/0af26894d1464b9bbb6a076952286374` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:02.955948 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af26894d1464b9bbb6a076952286374)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:02.957845 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.001320 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:03.008841 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af26894d1464b9bbb6a076952286374', 'request_id': 'req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.015384 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af26894d1464b9bbb6a076952286374', 'request_id': 'req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.021328 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af26894d1464b9bbb6a076952286374', 'request_id': 'req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.036641 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] The token cache is being invalidate because project 0af26894d1464b9bbb6a076952286374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:03.036933 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.037564 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.039355 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1438/2871] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:02 2022] DELETE /identity/v3/projects/0af26894d1464b9bbb6a076952286374 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:03.043172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.052975 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5589b90-ce82-44d5-bb18-249fa5fcfb1f service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.109759 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.110816 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.111174 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.111578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] PATH_INFO: `/v3/users/68e8bf7bde9c4935aa175dd83ef1a71f` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.112815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e8bf7bde9c4935aa175dd83ef1a71f)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.114741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.164321 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e8bf7bde9c4935aa175dd83ef1a71f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.173162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e8bf7bde9c4935aa175dd83ef1a71f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.185109 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e8bf7bde9c4935aa175dd83ef1a71f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.185747 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e8bf7bde9c4935aa175dd83ef1a71f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.192544 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-d457cdad-6085-4b64-a95c-d1f9be6674c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e8bf7bde9c4935aa175dd83ef1a71f'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.198568 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1434/2872] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:03 2022] DELETE /identity/v3/users/68e8bf7bde9c4935aa175dd83ef1a71f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:03.477549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.479046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:03.494519 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d6cd9-ce8c-4c61-bc49-2f1979697992 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.554746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.555565 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.555851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.556151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] PATH_INFO: `/v3/projects/a2dba3963fe643e8885aff33fb6a995c` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.557360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2dba3963fe643e8885aff33fb6a995c)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.559289 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.614965 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2dba3963fe643e8885aff33fb6a995c', 'request_id': 'req-0993f073-e1a8-466c-93d8-b99fca18e1ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.621960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2dba3963fe643e8885aff33fb6a995c', 'request_id': 'req-0993f073-e1a8-466c-93d8-b99fca18e1ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.628343 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2dba3963fe643e8885aff33fb6a995c', 'request_id': 'req-0993f073-e1a8-466c-93d8-b99fca18e1ea'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.641852 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] The token cache is being invalidate because project a2dba3963fe643e8885aff33fb6a995c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:03.642141 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.642777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-0993f073-e1a8-466c-93d8-b99fca18e1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:03.644376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1439/2873] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:03 2022] DELETE /identity/v3/projects/a2dba3963fe643e8885aff33fb6a995c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:03.649419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.649872 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.649954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.669225 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:03.670425 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:03.723537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:03.740886 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1435/2874] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:03.749155 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.749439 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.749712 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.768135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:03.769403 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:03.777406 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1440/2875] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:03.781902 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.792112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bcdd2-f113-4911-96ce-29f87cc1a071 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.797112 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.797763 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.798050 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.798312 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.799076 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:56:03.799393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.801022 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.812644 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1436/2876] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:56:03.816851 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.826067 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21409f18-ac3b-4029-99ae-466122fc663f None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.830761 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.831398 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.831709 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.831978 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.832664 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.834551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.862350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1441/2877] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:03 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:56:03.867114 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.880262 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2139b69-e981-4f55-a602-cad049894eec admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.938273 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.938945 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.939222 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.939582 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.940320 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.942438 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:03.964512 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1437/2878] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:03 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:56:03.970509 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:03.980897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf81b-ca89-49d2-b2cd-2d985803a72f admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:03.986831 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:03.987516 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:03.987818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:03.988074 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:03.988815 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:03.990433 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.001388 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1442/2879] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:04.005565 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.016947 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb1fa4-6979-49eb-822f-95e628aca016 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.022993 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.023698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.023976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.024256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.024997 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.026650 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e3f32-c5d2-4db2-b6cb-64a2c9ee6acd admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.038115 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1438/2880] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:56:04.042452 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.052492 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfce60-2df8-4730-bea3-bd9514a97efe admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.057393 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.058279 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.058567 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.058828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] PATH_INFO: `/v3/projects/3ee48a8daf514dfda6d7c67120b76073/users/25f6ca1c300a4381b9b2041a211474fe/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.073378 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee48a8daf514dfda6d7c67120b76073, user_id=25f6ca1c300a4381b9b2041a211474fe, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.076068 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.079149 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:04.092480 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1443/2881] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:04 2022] PUT /identity/v3/projects/3ee48a8daf514dfda6d7c67120b76073/users/25f6ca1c300a4381b9b2041a211474fe/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:04.103879 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.104165 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.104433 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.132227 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] MFA Rules not processed for user `25f6ca1c300a4381b9b2041a211474fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:04.133455 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:04.200422 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1439/2882] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:04.205368 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.289728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c755-d724-424e-9bed-cfff94c724ce admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.298860 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.299568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.299856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.300131 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.300818 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.302741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.330786 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1444/2883] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:04.335185 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.346740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8baec5-c537-43ad-8816-6db8102d2e1b None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.392370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:04.408906 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.409662 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.409936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.410207 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.410905 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.412884 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.434406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1440/2884] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:56:04.438681 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.449551 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd713fb-eee5-49e0-8897-44cab08472a0 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.454724 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.455441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.455703 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.455980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.456721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.458345 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.468700 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1445/2885] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:04.473016 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.483999 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d8ec9-37d8-4f79-abe6-f1f5d9a78bc4 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.490515 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.491172 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.491450 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.491753 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.492490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.494090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab75db5-8c04-4c33-8c1a-cc5d115da11e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.505039 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1441/2886] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:04.509317 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.519916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41adc1b-b414-4899-997c-062a60968a0e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.526122 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.526998 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.527272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.527549 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] PATH_INFO: `/v3/projects/4139c5d7a4594723a5b7df5b5edb3f08/users/cadb2ea452f74c32957e216fabb237d5/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.542907 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4139c5d7a4594723a5b7df5b5edb3f08, user_id=cadb2ea452f74c32957e216fabb237d5, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.545578 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.560360 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1446/2887] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:04 2022] PUT /identity/v3/projects/4139c5d7a4594723a5b7df5b5edb3f08/users/cadb2ea452f74c32957e216fabb237d5/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:04.570815 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.571100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.571371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.598155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] MFA Rules not processed for user `cadb2ea452f74c32957e216fabb237d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:04.599294 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:04.675037 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1442/2888] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:04.682030 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.755652 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30bcc6-d529-4bed-9ecf-a0eaf0407b6d admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.761868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.762646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.762925 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.763195 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.763912 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.765882 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.799830 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1447/2889] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:04.803258 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.814795 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e58670-80d5-4ff6-a274-58533a153041 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.874499 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.875107 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.875393 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.875701 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.876471 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.878442 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.901491 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1443/2890] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:04 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 12:56:04.906159 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.916721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0070301-42bb-45cf-86b2-b6fb7300b95c admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.921732 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.922392 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.922673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.922947 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.923801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.925422 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.936135 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1448/2891] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:04.940179 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:04.950718 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d60c-c35a-4c40-b451-0c41564e8cd2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:04.956133 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:04.957018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:04.957322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:04.957555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] PATH_INFO: `/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e/users/97d432d9fda44adea4cf8fa617856526/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:04.973267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ecee0df19e4800903bb7cc3e0b736e, user_id=97d432d9fda44adea4cf8fa617856526, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:04.975998 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:04.991152 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1444/2892] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:04 2022] PUT /identity/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e/users/97d432d9fda44adea4cf8fa617856526/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:56:04.995976 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:05.011327 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff428d25-d7f1-4bc0-9ef8-989f1259ab0e admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:05.031239 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:05.071781 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:05.072443 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:05.072721 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:05.072995 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:05.073719 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:05.075350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358224ca-a419-47b1-af70-2bad8d9e254c admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:05.085006 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1449/2893] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:05.089363 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:05.100296 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0be199-edc1-41c4-b48b-18b679dc3574 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:05.105965 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:05.106842 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:05.107139 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:05.107413 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] PATH_INFO: `/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e/users/97d432d9fda44adea4cf8fa617856526/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:05.110018 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ecee0df19e4800903bb7cc3e0b736e, user_id=97d432d9fda44adea4cf8fa617856526, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:05.113375 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:05.124740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1445/2894] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:05 2022] PUT /identity/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e/users/97d432d9fda44adea4cf8fa617856526/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:05.136157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:05.136464 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:05.136824 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:05.170460 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] MFA Rules not processed for user `97d432d9fda44adea4cf8fa617856526`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:05.171729 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:05.241258 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1450/2895] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:05.255189 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:05.257275 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:05.270035 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:05.328604 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d90fcc-1ec6-44fe-b224-d67db483f06f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:05.334774 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:05.335826 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:05.336271 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:05.336839 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:05.350148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:05.351626 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:05.361138 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1446/2896] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:05.405100 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:05.406794 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:05.420600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69cb48-0e8a-46ec-90f2-ca3bd41d9269 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:05.425312 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:05.425967 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:05.426228 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:05.426559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:05.486140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:05.487673 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8eff65-8743-413c-8364-fd6f0e5768cf service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:05.496955 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1451/2897] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:08.398940 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:08.468431 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c975790-348c-42c7-8efe-3ca374304311 service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:08.476371 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:08.477603 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:08.477977 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:08.478330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] PATH_INFO: `/v3/users/25f6ca1c300a4381b9b2041a211474fe` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:08.480267 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f6ca1c300a4381b9b2041a211474fe)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:08.483195 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:08.531949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f6ca1c300a4381b9b2041a211474fe'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:08.540432 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f6ca1c300a4381b9b2041a211474fe'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:08.552223 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f6ca1c300a4381b9b2041a211474fe'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:08.552843 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f6ca1c300a4381b9b2041a211474fe'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:08.559592 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f6ca1c300a4381b9b2041a211474fe'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:08.565841 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1447/2898] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:08 2022] DELETE /identity/v3/users/25f6ca1c300a4381b9b2041a211474fe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:08.570568 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:08.571055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:08.571164 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:08.589582 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:08.590801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:08.626510 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=100 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:08.653666 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1452/2899] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:08.664130 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:08.665578 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:08.698992 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:08.728567 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24dadeb-635d-4982-bacf-7bd7c53f160a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:08.732757 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:08.733406 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:08.733687 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:08.733956 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:08.743263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:08.744741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:08.754220 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1448/2900] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:09.050766 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:09.052318 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:09.065502 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe7bb-c3f6-40db-93a8-d06c22a628da None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:09.070494 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:09.071395 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:09.071930 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:09.072211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] PATH_INFO: `/v3/projects/3ee48a8daf514dfda6d7c67120b76073` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:09.073423 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee48a8daf514dfda6d7c67120b76073)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:09.075356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:09.122848 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee48a8daf514dfda6d7c67120b76073', 'request_id': 'req-6ea38af2-d789-48f4-9301-6a9502ffcac8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.129518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee48a8daf514dfda6d7c67120b76073', 'request_id': 'req-6ea38af2-d789-48f4-9301-6a9502ffcac8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.136365 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee48a8daf514dfda6d7c67120b76073', 'request_id': 'req-6ea38af2-d789-48f4-9301-6a9502ffcac8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.155278 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] The token cache is being invalidate because project 3ee48a8daf514dfda6d7c67120b76073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:09.155593 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.156283 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.157897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1453/2901] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:09 2022] DELETE /identity/v3/projects/3ee48a8daf514dfda6d7c67120b76073 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:09.162142 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:09.172341 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ae84-5654-4939-9c19-9035a4a5e07f service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:09.234091 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:09.234972 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:09.235250 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:09.235555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] PATH_INFO: `/v3/users/cadb2ea452f74c32957e216fabb237d5` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:09.236892 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadb2ea452f74c32957e216fabb237d5)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:09.239062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:09.297298 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadb2ea452f74c32957e216fabb237d5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.307784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadb2ea452f74c32957e216fabb237d5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.322027 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadb2ea452f74c32957e216fabb237d5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.322832 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadb2ea452f74c32957e216fabb237d5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.330960 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadb2ea452f74c32957e216fabb237d5'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.339865 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1449/2902] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:09 2022] DELETE /identity/v3/users/cadb2ea452f74c32957e216fabb237d5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:09.629667 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:09.631136 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:09.647499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38af2-d789-48f4-9301-6a9502ffcac8 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:09.708140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:09.709028 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:09.709310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:09.709587 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] PATH_INFO: `/v3/projects/4139c5d7a4594723a5b7df5b5edb3f08` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:09.710829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4139c5d7a4594723a5b7df5b5edb3f08)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:09.712809 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:09.762594 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4139c5d7a4594723a5b7df5b5edb3f08', 'request_id': 'req-871c614a-9c61-482b-aeb6-064dbc216db9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.768653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4139c5d7a4594723a5b7df5b5edb3f08', 'request_id': 'req-871c614a-9c61-482b-aeb6-064dbc216db9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.774371 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4139c5d7a4594723a5b7df5b5edb3f08', 'request_id': 'req-871c614a-9c61-482b-aeb6-064dbc216db9'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.777072 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:09.790679 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] The token cache is being invalidate because project 4139c5d7a4594723a5b7df5b5edb3f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:09.790969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.791630 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.793402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1454/2903] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:09 2022] DELETE /identity/v3/projects/4139c5d7a4594723a5b7df5b5edb3f08 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:09.797268 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:09.798732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:09.811847 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560ed3f-ab31-44e4-9c85-1906945fa74a admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:09.860551 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:09.872360 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:09.873230 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:09.873508 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:09.873784 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] PATH_INFO: `/v3/users/97d432d9fda44adea4cf8fa617856526` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:09.874978 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d432d9fda44adea4cf8fa617856526)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:09.876936 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:09.927735 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d432d9fda44adea4cf8fa617856526'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.935874 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d432d9fda44adea4cf8fa617856526'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.947739 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d432d9fda44adea4cf8fa617856526'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.948434 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d432d9fda44adea4cf8fa617856526'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.955182 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e1ec80b8-4610-4e8b-a681-7412a0c4a729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d432d9fda44adea4cf8fa617856526'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:09.961446 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1450/2904] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:09 2022] DELETE /identity/v3/users/97d432d9fda44adea4cf8fa617856526 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:10.246754 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.248493 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:10.261936 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c614a-9c61-482b-aeb6-064dbc216db9 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.319891 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.320765 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.321048 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.321325 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] PATH_INFO: `/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.322559 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ecee0df19e4800903bb7cc3e0b736e)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.324518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.376063 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ecee0df19e4800903bb7cc3e0b736e', 'request_id': 'req-2c8dd32c-d70a-45c9-9553-f661e7692996'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:10.382453 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ecee0df19e4800903bb7cc3e0b736e', 'request_id': 'req-2c8dd32c-d70a-45c9-9553-f661e7692996'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:10.388211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ecee0df19e4800903bb7cc3e0b736e', 'request_id': 'req-2c8dd32c-d70a-45c9-9553-f661e7692996'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:10.402227 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] The token cache is being invalidate because project a2ecee0df19e4800903bb7cc3e0b736e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:10.402513 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:10.403151 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2c8dd32c-d70a-45c9-9553-f661e7692996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:10.405079 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1455/2905] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:10 2022] DELETE /identity/v3/projects/a2ecee0df19e4800903bb7cc3e0b736e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:10.411369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.411705 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.411983 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.431000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:10.432256 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:10.493736 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1451/2906] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:10.502544 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.502906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.503193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.521856 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:10.523055 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:10.532056 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1456/2907] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:10.536813 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.547009 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db03d2a-1382-4055-baeb-ce4086f6ca89 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.552409 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.553063 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.553348 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.553624 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.554361 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:56:10.554675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.556313 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.568741 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1452/2908] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:56:10.574622 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.584342 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc27b87-6457-4aba-ab65-1be08937a8a2 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.590229 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.590913 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.591192 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.591461 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.592170 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.594088 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.622127 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1457/2909] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:10.626330 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.636537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5389c44-31c3-4cb0-b6bf-ce91b1d03c74 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.687441 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:10.696090 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.696778 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.697086 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.697369 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.698087 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.700594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.724825 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1453/2910] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:10.730914 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.741979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006dda-c4fa-48b3-93f5-5b02fd6d44dc admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.747986 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.748653 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.748940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.749211 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.749945 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.751596 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.762094 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1458/2911] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:10.766286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.776796 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5241a-0fca-4c9a-b425-7dd6aaf9c9da admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.782370 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.783066 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.783351 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.783675 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.784461 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.786119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b1714-ff15-46c5-9ef2-e643f9d78073 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.796386 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1454/2912] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:10.800623 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.810940 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df7379-fe84-4372-b06a-48072f12e592 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:10.816521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:10.817402 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.817682 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.817956 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] PATH_INFO: `/v3/projects/44ecca23b34a403bbd9d1d7676ee9653/users/d42228159ed64422bd377ab95b446bde/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.833276 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ecca23b34a403bbd9d1d7676ee9653, user_id=d42228159ed64422bd377ab95b446bde, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:10.835980 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:10.850993 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1459/2913] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:10 2022] PUT /identity/v3/projects/44ecca23b34a403bbd9d1d7676ee9653/users/d42228159ed64422bd377ab95b446bde/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:10.862633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:10.863126 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:10.863211 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:10.890528 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] MFA Rules not processed for user `d42228159ed64422bd377ab95b446bde`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:10.891711 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:10.964671 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1455/2914] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:10.969290 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:10.990629 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:11.033928 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536176-41d3-4b9e-b9bb-209511ae8277 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.039310 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.039987 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.040272 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.040545 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.041240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.043157 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.072145 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1460/2915] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:10 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:11.078404 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.090696 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4f3b-88f6-4fab-8608-dfb32f0c38f3 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.148136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.148817 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.149100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.149376 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.150056 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.152025 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.173897 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1456/2916] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:11 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:11.178042 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.188741 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c462-abc6-4c57-bf8f-4b059a087557 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.194810 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.195497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.195793 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.196070 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.196805 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.198437 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.209580 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1461/2917] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:11.213666 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.224543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e098df-96cf-46f2-b16f-4e507de9e5e2 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.230698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.231633 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.231908 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.232187 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] PATH_INFO: `/v3/projects/7e600888f98544e7849f74879ab5c0e0/users/92c9b770f55a47c6972586f6953804e9/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.249021 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e600888f98544e7849f74879ab5c0e0, user_id=92c9b770f55a47c6972586f6953804e9, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.251779 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.266849 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1457/2918] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:11 2022] PUT /identity/v3/projects/7e600888f98544e7849f74879ab5c0e0/users/92c9b770f55a47c6972586f6953804e9/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:11.270938 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.281906 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bb065-18b0-4cd4-ba58-1154a8260016 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.344374 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.345084 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.345372 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.345640 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.346350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.348037 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736faf8-8932-4105-808c-5769f72fa44e admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.358424 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1462/2919] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 12:56:11.362643 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.373118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814b4a0-a72f-4f78-b42f-638ab54f627f admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.380176 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.381555 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.381984 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.382395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] PATH_INFO: `/v3/projects/7e600888f98544e7849f74879ab5c0e0/users/92c9b770f55a47c6972586f6953804e9/roles/abe682bd8c304ac3993415ae5ef145ee` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.385325 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e600888f98544e7849f74879ab5c0e0, user_id=92c9b770f55a47c6972586f6953804e9, role_id=abe682bd8c304ac3993415ae5ef145ee)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.389613 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.405373 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1458/2920] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:11 2022] PUT /identity/v3/projects/7e600888f98544e7849f74879ab5c0e0/users/92c9b770f55a47c6972586f6953804e9/roles/abe682bd8c304ac3993415ae5ef145ee => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:11.419250 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.419778 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.420262 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.455462 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] MFA Rules not processed for user `92c9b770f55a47c6972586f6953804e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:11.456791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:11.528455 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1463/2921] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:11.540160 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:11.542263 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:11.564426 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:11.601205 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa318cb-385a-4e55-b4d5-f4ffb59f8ee6 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:11.605136 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:11.605777 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:11.606052 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:11.606335 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:11.664252 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:11.665632 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:11.674495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1459/2922] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:15.939923 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:15.941528 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:15.954090 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d6dd-a20b-402d-b8b5-fdfe90a1f50d None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:15.958075 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:15.958801 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:15.959077 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:15.959347 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:15.969369 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:15.970791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491b219-b836-4346-8544-85cb35bf66c8 service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:15.980350 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1464/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:20.933561 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:21.009660 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1b8cc-5271-419b-a86b-84d71c16fc1b service cinder] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:21.015148 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:21.016053 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:21.016338 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:21.016619 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] PATH_INFO: `/v3/users/d42228159ed64422bd377ab95b446bde` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:21.017889 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42228159ed64422bd377ab95b446bde)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:21.019856 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:21.068322 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42228159ed64422bd377ab95b446bde'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.075976 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42228159ed64422bd377ab95b446bde'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.087698 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42228159ed64422bd377ab95b446bde'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.088315 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42228159ed64422bd377ab95b446bde'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.090726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:21.097281 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42228159ed64422bd377ab95b446bde'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.104216 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1460/2924] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:20 2022] DELETE /identity/v3/users/d42228159ed64422bd377ab95b446bde => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:21.109020 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:21.109497 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:21.109600 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:21.115588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:21.129890 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:21.131089 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:21.192952 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1465/2925] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 12:56:21.203059 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:21.204579 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:21.265961 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7232-25a2-450b-a116-f15f94f40f67 admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:21.270954 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:21.271653 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:21.271929 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:21.272221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:21.282654 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:21.284127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:21.293732 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1461/2926] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:56:21.567356 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:21.568823 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:21.581920 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0447831-9f11-4fbd-be4a-66a9bb52b9a0 None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:21.586901 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:21.587822 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:21.588107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:21.588427 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] PATH_INFO: `/v3/projects/44ecca23b34a403bbd9d1d7676ee9653` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:21.589728 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ecca23b34a403bbd9d1d7676ee9653)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:21.591916 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:21.639387 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ecca23b34a403bbd9d1d7676ee9653', 'request_id': 'req-2994ad6d-d596-4324-a927-215affa68e93'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.645521 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ecca23b34a403bbd9d1d7676ee9653', 'request_id': 'req-2994ad6d-d596-4324-a927-215affa68e93'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.651542 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ecca23b34a403bbd9d1d7676ee9653', 'request_id': 'req-2994ad6d-d596-4324-a927-215affa68e93'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.665871 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] The token cache is being invalidate because project 44ecca23b34a403bbd9d1d7676ee9653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:21.666275 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.666897 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.668499 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1466/2927] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:21 2022] DELETE /identity/v3/projects/44ecca23b34a403bbd9d1d7676ee9653 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:21.672715 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:21.682479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f2567-4c3b-4778-b14a-f3b8add5b80d service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:21.744155 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:21.744980 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:21.745236 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:21.745537 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] PATH_INFO: `/v3/users/92c9b770f55a47c6972586f6953804e9` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:21.746773 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c9b770f55a47c6972586f6953804e9)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:21.748850 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:21.800460 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c9b770f55a47c6972586f6953804e9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.809082 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c9b770f55a47c6972586f6953804e9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.821100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c9b770f55a47c6972586f6953804e9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.821740 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c9b770f55a47c6972586f6953804e9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.829213 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-2dd7abde-ee27-46f9-90ee-b9d27a699cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c9b770f55a47c6972586f6953804e9'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:21.835580 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1462/2928] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:21 2022] DELETE /identity/v3/users/92c9b770f55a47c6972586f6953804e9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 12:56:22.128119 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.129611 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:22.143057 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994ad6d-d596-4324-a927-215affa68e93 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.202193 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.203047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.203331 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.203627 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] PATH_INFO: `/v3/projects/7e600888f98544e7849f74879ab5c0e0` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.204799 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e600888f98544e7849f74879ab5c0e0)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.206710 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.238043 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:22.254471 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e600888f98544e7849f74879ab5c0e0', 'request_id': 'req-55c98438-536d-4b88-9129-dec604462e70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:22.260836 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e600888f98544e7849f74879ab5c0e0', 'request_id': 'req-55c98438-536d-4b88-9129-dec604462e70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:22.266560 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e600888f98544e7849f74879ab5c0e0', 'request_id': 'req-55c98438-536d-4b88-9129-dec604462e70'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:22.278867 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] The token cache is being invalidate because project 7e600888f98544e7849f74879ab5c0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:22.279156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:22.280191 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-55c98438-536d-4b88-9129-dec604462e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:22.281760 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1467/2929] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:22 2022] DELETE /identity/v3/projects/7e600888f98544e7849f74879ab5c0e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:22.287875 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.288365 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.288453 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.307199 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:22.308514 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:22.317419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:22.369867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1463/2930] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:22.378446 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.378777 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.379060 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.398148 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:22.399335 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:22.407900 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1468/2931] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:22.412062 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.421859 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00d39a-ffea-4cf8-bbaf-105a95b5b1e1 None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.426640 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.427286 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.427593 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.427864 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] PATH_INFO: `/v3/domains` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.428596 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80383) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 12:56:22.428901 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.430486 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.442326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1464/2932] 10.222.0.18 () {58 vars in 1152 bytes} [Thu Jul 21 12:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 12:56:22.446604 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.456171 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b36e-0082-4f24-96d7-f390836a10db None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.460960 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.461588 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.461886 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.462156 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.462868 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.464828 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.494240 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1469/2933] 10.222.0.18 () {60 vars in 1152 bytes} [Thu Jul 21 12:56:22 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:22.499014 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.513135 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06566262-0400-401b-9927-5f7da9a8f33e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.573436 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.574162 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.574463 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.574737 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] PATH_INFO: `/v3/users` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.575419 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.577359 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.598490 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1465/2934] 10.222.0.18 () {60 vars in 1143 bytes} [Thu Jul 21 12:56:22 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 12:56:22.602767 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.613358 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead151e-ff3f-43d4-a0fc-136edb932328 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.618179 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.618829 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.619107 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.619377 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.620139 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.621791 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.632047 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1470/2935] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:22.635766 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.645496 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37923-e4c9-4079-b106-49780bc976fa admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.650221 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.650867 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.651141 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.651412 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.652228 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.653827 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de4966-8714-41c6-8110-844486623aeb admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.664201 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1466/2936] 10.222.0.18 () {58 vars in 1121 bytes} [Thu Jul 21 12:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 12:56:22.668440 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.678969 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ef5b8-b622-4be7-8a0d-8a08051a2791 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.684825 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.685720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.685979 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.686251 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] PATH_INFO: `/v3/projects/ed826c7196f547068dc6762d1247bf2f/users/c50575bbf1ba49a1892a305fc56635a7/roles/f7a7a1e9e73d449ea34fa43e31b197f6` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.701735 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed826c7196f547068dc6762d1247bf2f, user_id=c50575bbf1ba49a1892a305fc56635a7, role_id=f7a7a1e9e73d449ea34fa43e31b197f6)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.704420 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.719002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1471/2937] 10.222.0.18 () {60 vars in 1482 bytes} [Thu Jul 21 12:56:22 2022] PUT /identity/v3/projects/ed826c7196f547068dc6762d1247bf2f/users/c50575bbf1ba49a1892a305fc56635a7/roles/f7a7a1e9e73d449ea34fa43e31b197f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:22.730594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] REQUEST_METHOD: `POST` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.730881 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.731156 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.759473 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.auth.core [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] MFA Rules not processed for user `c50575bbf1ba49a1892a305fc56635a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80383) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:22.760725 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:22.824595 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1467/2938] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:22.836899 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:22.838746 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:22.901864 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642bab8-8855-4214-b9c0-6da17971fff1 admin admin] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:22.906451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5493d9cde9404225bedc423f0ecf213c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:22.907097 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] REQUEST_METHOD: `GET` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:22.907376 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:22.907684 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:22.916451 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:22.917944 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa0dc1-9576-48bb-bc55-6a4a5a0f64be service cinder] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:22.926959 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1472/2939] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:56:31.080550 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:31.091521 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:31.149678 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab649-dd45-40c0-a9e4-2c648eacaf0a None None] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:31.155367 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:31.156301 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:31.156585 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:31.156861 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] PATH_INFO: `/v3/users/c50575bbf1ba49a1892a305fc56635a7` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:31.158127 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50575bbf1ba49a1892a305fc56635a7)` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:31.160100 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:31.208852 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50575bbf1ba49a1892a305fc56635a7'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.216479 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50575bbf1ba49a1892a305fc56635a7'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.230280 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50575bbf1ba49a1892a305fc56635a7'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.230880 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50575bbf1ba49a1892a305fc56635a7'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.238214 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.notifications [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50575bbf1ba49a1892a305fc56635a7'} {{(pid=80383) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.244663 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1468/2940] 10.222.0.18 () {58 vars in 1223 bytes} [Thu Jul 21 12:56:31 2022] DELETE /identity/v3/users/c50575bbf1ba49a1892a305fc56635a7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:31.249525 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] REQUEST_METHOD: `POST` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:31.250002 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:31.250112 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:31.267714 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.auth.core [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] MFA Rules not processed for user `7aeac511a0044d6781fee6120328a71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 12:56:31.268896 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:31.317646 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:31.331638 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1473/2941] 10.222.0.18 () {58 vars in 957 bytes} [Thu Jul 21 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 12:56:31.341594 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:31.343036 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:31.398495 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b5b46-610b-4847-9b71-5fec18595d8e admin admin] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:31.402308 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcb115c5cad84f55b143c392c8c23dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:31.402949 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:31.403224 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:31.403507 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:31.413119 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:31.414543 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:31.424621 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1469/2942] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 12:56:31.707736 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] Authenticating user token {{(pid=80384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:56:31.709341 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.fernet_utils [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:56:31.723518 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8da351-5578-401b-ac69-a79bff30936c None None] Validating token access rules against request {{(pid=80384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:56:31.729214 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aeac511a0044d6781fee6120328a71c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a4579ac3a4640e986aabfbb2b1ede46', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:56:31.730140 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:56:31.730441 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:56:31.730720 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] PATH_INFO: `/v3/projects/ed826c7196f547068dc6762d1247bf2f` {{(pid=80384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:56:31.732247 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed826c7196f547068dc6762d1247bf2f)` {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:56:31.734224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] RBAC: Authorization granted {{(pid=80384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:56:31.784189 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed826c7196f547068dc6762d1247bf2f', 'request_id': 'req-327d5768-e6af-41f2-b4e4-82fe4c5677b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.790224 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed826c7196f547068dc6762d1247bf2f', 'request_id': 'req-327d5768-e6af-41f2-b4e4-82fe4c5677b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.796178 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed826c7196f547068dc6762d1247bf2f', 'request_id': 'req-327d5768-e6af-41f2-b4e4-82fe4c5677b8'} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.809120 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] The token cache is being invalidate because project ed826c7196f547068dc6762d1247bf2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 12:56:31.809412 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.810046 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG keystone.notifications [None req-327d5768-e6af-41f2-b4e4-82fe4c5677b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 12:56:31.811773 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: [pid: 80384|app: 0|req: 1474/2943] 10.222.0.18 () {58 vars in 1232 bytes} [Thu Jul 21 12:56:31 2022] DELETE /identity/v3/projects/ed826c7196f547068dc6762d1247bf2f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 12:56:41.420118 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:56:41.806231 dv-cinder-850321-1-iscsi devstack@keystone.service[80384]: DEBUG dbcounter [-] [80384] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 12:59:18.298707 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] Authenticating user token {{(pid=80383) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 12:59:18.300395 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:59:18.373684 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59624a-fbe0-4a3f-9a59-67da40c1226f service neutron] Validating token access rules against request {{(pid=80383) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 12:59:18.436581 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76fd650362ab4b4f959c08d46e1827ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a4db6abd1a244249ec2954699f21c5b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80383) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 12:59:18.437235 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] REQUEST_METHOD: `GET` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 12:59:18.437517 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] SCRIPT_NAME: `/identity` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 12:59:18.437764 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80383) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 12:59:18.439862 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.fernet_utils [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80383) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 12:59:18.452726 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 12:59:18.454326 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341dee4-cdd9-4744-af54-ac8b338be5ec service placement] RBAC: Authorization granted {{(pid=80383) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 12:59:18.465000 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: [pid: 80383|app: 0|req: 1470/2944] 10.222.0.18 () {60 vars in 1365 bytes} [Thu Jul 21 12:59:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 12:59:28.460771 dv-cinder-850321-1-iscsi devstack@keystone.service[80383]: DEBUG dbcounter [-] [80383] Writing DB stats keystone:SELECT=58 {{(pid=80383) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}